2025-11-13 09:06:50,126 [ 254697 ] INFO : ClickHouse root is not set. Will use /home/ubuntu/_work/ClickHouse/ClickHouse (runner:53, check_args_and_update_paths) 2025-11-13 09:06:50,127 [ 254697 ] INFO : Cases dir is not set. Will use /home/ubuntu/_work/ClickHouse/ClickHouse/tests/integration (runner:79, check_args_and_update_paths) 2025-11-13 09:06:50,127 [ 254697 ] INFO : utils dir is not set. Will use /home/ubuntu/_work/ClickHouse/ClickHouse/utils (runner:90, check_args_and_update_paths) 2025-11-13 09:06:50,127 [ 254697 ] INFO : base_configs_dir: /home/ubuntu/_work/ClickHouse/ClickHouse/programs/server, binary: /home/ubuntu/_work/_temp/test/build/clickhouse, cases_dir: /home/ubuntu/_work/ClickHouse/ClickHouse/tests/integration (runner:92, check_args_and_update_paths) clickhouse_integration_tests_volume Running pytest container as: 'docker run --rm --name clickhouse_integration_tests_94hpeb --privileged --dns-search='.' --memory=30709030912 --security-opt seccomp=unconfined --cap-add=SYS_PTRACE --volume=/home/ubuntu/_work/_temp/test/build/clickhouse:/clickhouse --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/programs/server:/clickhouse-config --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/tests/integration:/ClickHouse/tests/integration --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/utils/backupview:/ClickHouse/utils/backupview --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/utils/grpc-client/pb2:/ClickHouse/utils/grpc-client/pb2 --volume=/run:/run/host:ro --volume=clickhouse_integration_tests_volume:/var/lib/docker -e DOCKER_DOTNET_CLIENT_TAG=11de0b29a15d -e DOCKER_HELPER_TAG=5dc43a6382f0 -e DOCKER_BASE_TAG=5ccda723c1fc -e DOCKER_KERBEROS_KDC_TAG=9391ecdee8d7 -e DOCKER_MYSQL_GOLANG_CLIENT_TAG=9bec2a638e6e -e DOCKER_MYSQL_JAVA_CLIENT_TAG=766bff31cfe4 -e DOCKER_MYSQL_JS_CLIENT_TAG=41ba7c2ec2a1 -e DOCKER_MYSQL_PHP_CLIENT_TAG=88be89c1e3b6 -e DOCKER_NGINX_DAV_TAG=b55ac9cd7519 -e DOCKER_POSTGRESQL_JAVA_CLIENT_TAG=a4eff5c7f4d6 -e DOCKER_PYTHON_BOTTLE_TAG=d862517635bf -e DOCKER_CLIENT_TIMEOUT=300 -e COMPOSE_HTTP_TIMEOUT=600 -e PYTHONUNBUFFERED=1 -e PYTEST_ADDOPTS="--dist=loadfile -n 10 -rfEps --run-id=0 --color=no --durations=0 --report-log=parallel1_0.jsonl --report-log-exclude-logs-on-passed-tests test_executable_dictionary/test.py::test_executable_input_bash test_executable_dictionary/test.py::test_executable_input_python test_executable_dictionary/test.py::test_executable_input_send_chunk_header_python test_executable_dictionary/test.py::test_executable_input_signalled_python test_executable_dictionary/test.py::test_executable_input_slow_python test_executable_dictionary/test.py::test_executable_input_sum_python test_executable_dictionary/test.py::test_executable_non_direct_input_bash test_executable_dictionary/test.py::test_executable_source_argument_python test_executable_dictionary/test.py::test_executable_source_exit_code_check test_executable_dictionary/test.py::test_executable_source_python test_executable_dictionary/test.py::test_executable_source_updated_python test_format_avro_confluent/test.py::test_select test_format_avro_confluent/test.py::test_select_auth test_format_avro_confluent/test.py::test_select_auth_encoded test_format_avro_confluent/test.py::test_select_auth_encoded_complex test_grant_and_revoke/test_without_table_engine_grant.py::test_table_engine_and_source_grant test_groupBitmapAnd_on_distributed/test_groupBitmapAndState_on_distributed_table.py::test_groupBitmapAndState_on_distributed_table test_keeper_profiler/test.py::test_profiler test_keeper_reconfig_remove_many/test.py::test_reconfig_remove_2_and_leader test_keeper_restore_from_snapshot/test_disk_s3.py::test_recover_from_snapshot_with_disk_s3 test_keeper_snapshot_small_distance/test.py::test_snapshot_and_load test_kerberos_auth/test.py::test_bad_path_to_keytab test_kerberos_auth/test.py::test_kerberos_auth_with_keytab test_kerberos_auth/test.py::test_kerberos_auth_without_keytab test_log_levels_update/test.py::test_log_levels_update test_log_query_probability/test.py::test_log_quries_probability_one test_log_query_probability/test.py::test_log_quries_probability_two test_match_process_uid_against_data_owner/test.py::test_different_user test_max_authentication_methods_per_user/test.py::test_alter test_max_authentication_methods_per_user/test.py::test_alter_default_setting test_max_authentication_methods_per_user/test.py::test_create test_max_authentication_methods_per_user/test.py::test_create_default_setting test_merge_tree_check_part_with_cache/test.py::test_check_part_with_cache 'test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_distributed[test_multiple_shards_multiple_replicas-custom_key_sampling-key]' 'test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_distributed[test_multiple_shards_multiple_replicas-custom_key_sampling-sipHash64(key)]' 'test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_distributed[test_single_shard_multiple_replicas-custom_key_sampling-key]' 'test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_distributed[test_single_shard_multiple_replicas-custom_key_sampling-sipHash64(key)]' 'test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_mergetree[test_single_shard_multiple_replicas-custom_key_range-key]' 'test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_mergetree[test_single_shard_multiple_replicas-custom_key_range-sipHash64(key)]' 'test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_mergetree[test_single_shard_multiple_replicas-custom_key_sampling-key]' 'test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_mergetree[test_single_shard_multiple_replicas-custom_key_sampling-sipHash64(key)]' 'test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_replicatedmergetree[test_single_shard_multiple_replicas-custom_key_range-key]' 'test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_replicatedmergetree[test_single_shard_multiple_replicas-custom_key_range-sipHash64(key)]' 'test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_replicatedmergetree[test_single_shard_multiple_replicas-custom_key_sampling-key]' 'test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_replicatedmergetree[test_single_shard_multiple_replicas-custom_key_sampling-sipHash64(key)]' test_parallel_replicas_increase_error_count/test.py::test_increase_error_count 'test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-101-1000-SELECT sum(key) FROM {table_name}]' 'test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-101-10000-SELECT sum(key) FROM {table_name}]' 'test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-101-100000-SELECT sum(key) FROM {table_name}]' 'test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-11-1000-SELECT sum(key) FROM {table_name}]' 'test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-11-10000-SELECT sum(key) FROM {table_name}]' 'test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-11-100000-SELECT sum(key) FROM {table_name}]' 'test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-101-1000-SELECT sum(key) FROM {table_name}]' 'test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-101-10000-SELECT sum(key) FROM {table_name}]' 'test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-101-100000-SELECT sum(key) FROM {table_name}]' 'test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-11-1000-SELECT sum(key) FROM {table_name}]' 'test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-11-10000-SELECT sum(key) FROM {table_name}]' 'test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-11-100000-SELECT sum(key) FROM {table_name}]' test_password_constraints/test.py::test_complexity_rules test_password_constraints/test.py::test_default_password_type test_polymorphic_parts/test.py::test_compact_parts_only 'test_polymorphic_parts/test.py::test_different_part_types_on_replicas[polymorphic_table_compact-Compact]' 'test_polymorphic_parts/test.py::test_different_part_types_on_replicas[polymorphic_table_wide-Wide]' 'test_polymorphic_parts/test.py::test_polymorphic_parts_basics[first_node0-second_node0]' 'test_polymorphic_parts/test.py::test_polymorphic_parts_basics[first_node1-second_node1]' test_polymorphic_parts/test.py::test_polymorphic_parts_index test_polymorphic_parts/test.py::test_polymorphic_parts_non_adaptive test_profile_events_s3/test.py::test_profile_events 'test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-None-False-False]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-None-False-True]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-None-True-False]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-None-True-True]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-depends_on1-False-False]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-depends_on1-False-True]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-depends_on1-True-False]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-depends_on1-True-True]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-None-False-False]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-None-False-True]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-None-True-False]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-None-True-True]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-depends_on1-False-False]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-depends_on1-False-True]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-depends_on1-True-False]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-depends_on1-True-True]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-None-False-False]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-None-False-True]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-None-True-False]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-None-True-True]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-depends_on1-False-False]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-depends_on1-False-True]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-depends_on1-True-False]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-depends_on1-True-True]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-None-False-False]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-None-False-True]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-None-True-False]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-None-True-True]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-depends_on1-False-False]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-depends_on1-False-True]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-depends_on1-True-False]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-depends_on1-True-True]' -vvv " altinityinfra/integration-tests-runner:226bfaf75ac1 '. Start tests ============================= test session starts ============================== platform linux -- Python 3.10.12, pytest-7.4.4, pluggy-1.5.0 -- /usr/bin/python3 cachedir: .pytest_cache Test order randomisation NOT enabled. Enable with --random-order or --random-order-bucket= rootdir: /ClickHouse/tests/integration configfile: pytest.ini plugins: timeout-2.3.1, repeat-0.9.3, order-1.0.0, reportlog-0.4.0, xdist-3.5.0, random-order-1.1.1 timeout: 900.0s timeout method: signal timeout func_only: False created: 10/10 workers 10 workers [100 items] scheduling tests via LoadFileScheduling test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-None-False-False] test_kerberos_auth/test.py::test_bad_path_to_keytab test_format_avro_confluent/test.py::test_select test_log_query_probability/test.py::test_log_quries_probability_one test_password_constraints/test.py::test_complexity_rules test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-101-1000-SELECT sum(key) FROM {table_name}] test_polymorphic_parts/test.py::test_compact_parts_only test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_distributed[test_multiple_shards_multiple_replicas-custom_key_sampling-key] test_max_authentication_methods_per_user/test.py::test_alter test_executable_dictionary/test.py::test_executable_input_bash [gw5] [ 1%] PASSED test_max_authentication_methods_per_user/test.py::test_alter test_max_authentication_methods_per_user/test.py::test_alter_default_setting [gw6] [ 2%] PASSED test_kerberos_auth/test.py::test_bad_path_to_keytab test_kerberos_auth/test.py::test_kerberos_auth_with_keytab [gw6] [ 3%] PASSED test_kerberos_auth/test.py::test_kerberos_auth_with_keytab test_kerberos_auth/test.py::test_kerberos_auth_without_keytab [gw9] [ 4%] PASSED test_password_constraints/test.py::test_complexity_rules test_password_constraints/test.py::test_default_password_type [gw6] [ 5%] PASSED test_kerberos_auth/test.py::test_kerberos_auth_without_keytab [gw9] [ 6%] PASSED test_password_constraints/test.py::test_default_password_type [gw5] [ 7%] PASSED test_max_authentication_methods_per_user/test.py::test_alter_default_setting test_max_authentication_methods_per_user/test.py::test_create [gw5] [ 8%] PASSED test_max_authentication_methods_per_user/test.py::test_create test_max_authentication_methods_per_user/test.py::test_create_default_setting [gw5] [ 9%] PASSED test_max_authentication_methods_per_user/test.py::test_create_default_setting [gw3] [ 10%] PASSED test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_distributed[test_multiple_shards_multiple_replicas-custom_key_sampling-key] test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_distributed[test_multiple_shards_multiple_replicas-custom_key_sampling-sipHash64(key)] test_groupBitmapAnd_on_distributed/test_groupBitmapAndState_on_distributed_table.py::test_groupBitmapAndState_on_distributed_table [gw7] [ 11%] PASSED test_executable_dictionary/test.py::test_executable_input_bash test_executable_dictionary/test.py::test_executable_input_python [gw7] [ 12%] PASSED test_executable_dictionary/test.py::test_executable_input_python test_executable_dictionary/test.py::test_executable_input_send_chunk_header_python [gw7] [ 13%] PASSED test_executable_dictionary/test.py::test_executable_input_send_chunk_header_python test_executable_dictionary/test.py::test_executable_input_signalled_python [gw3] [ 14%] PASSED test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_distributed[test_multiple_shards_multiple_replicas-custom_key_sampling-sipHash64(key)] test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_distributed[test_single_shard_multiple_replicas-custom_key_sampling-key] [gw7] [ 15%] PASSED test_executable_dictionary/test.py::test_executable_input_signalled_python test_executable_dictionary/test.py::test_executable_input_slow_python test_keeper_snapshot_small_distance/test.py::test_snapshot_and_load [gw3] [ 16%] PASSED test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_distributed[test_single_shard_multiple_replicas-custom_key_sampling-key] test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_distributed[test_single_shard_multiple_replicas-custom_key_sampling-sipHash64(key)] [gw0] [ 17%] PASSED test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-101-1000-SELECT sum(key) FROM {table_name}] test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-101-10000-SELECT sum(key) FROM {table_name}] [gw3] [ 18%] PASSED test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_distributed[test_single_shard_multiple_replicas-custom_key_sampling-sipHash64(key)] test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_mergetree[test_single_shard_multiple_replicas-custom_key_range-key] [gw2] [ 19%] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-None-False-False] [gw7] [ 20%] PASSED test_executable_dictionary/test.py::test_executable_input_slow_python test_executable_dictionary/test.py::test_executable_input_sum_python test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-None-False-True] [gw7] [ 21%] PASSED test_executable_dictionary/test.py::test_executable_input_sum_python test_executable_dictionary/test.py::test_executable_non_direct_input_bash [gw3] [ 22%] PASSED test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_mergetree[test_single_shard_multiple_replicas-custom_key_range-key] test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_mergetree[test_single_shard_multiple_replicas-custom_key_range-sipHash64(key)] [gw7] [ 23%] PASSED test_executable_dictionary/test.py::test_executable_non_direct_input_bash test_executable_dictionary/test.py::test_executable_source_argument_python [gw1] [ 24%] PASSED test_format_avro_confluent/test.py::test_select test_format_avro_confluent/test.py::test_select_auth test_keeper_profiler/test.py::test_profiler [gw7] [ 25%] PASSED test_executable_dictionary/test.py::test_executable_source_argument_python test_executable_dictionary/test.py::test_executable_source_exit_code_check [gw1] [ 26%] PASSED test_format_avro_confluent/test.py::test_select_auth test_format_avro_confluent/test.py::test_select_auth_encoded [gw1] [ 27%] PASSED test_format_avro_confluent/test.py::test_select_auth_encoded test_format_avro_confluent/test.py::test_select_auth_encoded_complex [gw3] [ 28%] PASSED test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_mergetree[test_single_shard_multiple_replicas-custom_key_range-sipHash64(key)] test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_mergetree[test_single_shard_multiple_replicas-custom_key_sampling-key] [gw1] [ 29%] PASSED test_format_avro_confluent/test.py::test_select_auth_encoded_complex [gw7] [ 30%] PASSED test_executable_dictionary/test.py::test_executable_source_exit_code_check test_executable_dictionary/test.py::test_executable_source_python [gw2] [ 31%] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-None-False-True] test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-None-True-False] [gw7] [ 32%] PASSED test_executable_dictionary/test.py::test_executable_source_python test_executable_dictionary/test.py::test_executable_source_updated_python [gw3] [ 33%] PASSED test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_mergetree[test_single_shard_multiple_replicas-custom_key_sampling-key] test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_mergetree[test_single_shard_multiple_replicas-custom_key_sampling-sipHash64(key)] [gw9] [ 34%] PASSED test_groupBitmapAnd_on_distributed/test_groupBitmapAndState_on_distributed_table.py::test_groupBitmapAndState_on_distributed_table [gw0] [ 35%] PASSED test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-101-10000-SELECT sum(key) FROM {table_name}] test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-101-100000-SELECT sum(key) FROM {table_name}] [gw6] [ 36%] PASSED test_keeper_profiler/test.py::test_profiler [gw3] [ 37%] PASSED test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_mergetree[test_single_shard_multiple_replicas-custom_key_sampling-sipHash64(key)] test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_replicatedmergetree[test_single_shard_multiple_replicas-custom_key_range-key] [gw8] [ 38%] PASSED test_log_query_probability/test.py::test_log_quries_probability_one test_log_query_probability/test.py::test_log_quries_probability_two [gw2] [ 39%] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-None-True-False] test_keeper_restore_from_snapshot/test_disk_s3.py::test_recover_from_snapshot_with_disk_s3 [gw7] [ 40%] PASSED test_executable_dictionary/test.py::test_executable_source_updated_python test_profile_events_s3/test.py::test_profile_events test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-None-True-True] [gw4] [ 41%] PASSED test_polymorphic_parts/test.py::test_compact_parts_only test_polymorphic_parts/test.py::test_different_part_types_on_replicas[polymorphic_table_compact-Compact] [gw3] [ 42%] PASSED test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_replicatedmergetree[test_single_shard_multiple_replicas-custom_key_range-key] test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_replicatedmergetree[test_single_shard_multiple_replicas-custom_key_range-sipHash64(key)] [gw4] [ 43%] PASSED test_polymorphic_parts/test.py::test_different_part_types_on_replicas[polymorphic_table_compact-Compact] test_polymorphic_parts/test.py::test_different_part_types_on_replicas[polymorphic_table_wide-Wide] [gw2] [ 44%] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-None-True-True] test_keeper_reconfig_remove_many/test.py::test_reconfig_remove_2_and_leader test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-depends_on1-False-False] [gw3] [ 45%] PASSED test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_replicatedmergetree[test_single_shard_multiple_replicas-custom_key_range-sipHash64(key)] test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_replicatedmergetree[test_single_shard_multiple_replicas-custom_key_sampling-key] [gw4] [ 46%] PASSED test_polymorphic_parts/test.py::test_different_part_types_on_replicas[polymorphic_table_wide-Wide] test_polymorphic_parts/test.py::test_polymorphic_parts_basics[first_node0-second_node0] [gw3] [ 47%] PASSED test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_replicatedmergetree[test_single_shard_multiple_replicas-custom_key_sampling-key] test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_replicatedmergetree[test_single_shard_multiple_replicas-custom_key_sampling-sipHash64(key)] [gw0] [ 48%] PASSED test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-101-100000-SELECT sum(key) FROM {table_name}] test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-11-1000-SELECT sum(key) FROM {table_name}] [gw2] [ 49%] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-depends_on1-False-False] test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-depends_on1-False-True] [gw3] [ 50%] PASSED test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_replicatedmergetree[test_single_shard_multiple_replicas-custom_key_sampling-sipHash64(key)] [gw2] [ 51%] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-depends_on1-False-True] test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-depends_on1-True-False] [gw6] [ 52%] FAILED test_keeper_reconfig_remove_many/test.py::test_reconfig_remove_2_and_leader [gw1] [ 53%] PASSED test_profile_events_s3/test.py::test_profile_events [gw2] [ 54%] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-depends_on1-True-False] test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-depends_on1-True-True] [gw9] [ 55%] PASSED test_keeper_restore_from_snapshot/test_disk_s3.py::test_recover_from_snapshot_with_disk_s3 [gw0] [ 56%] PASSED test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-11-1000-SELECT sum(key) FROM {table_name}] test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-11-10000-SELECT sum(key) FROM {table_name}] [gw2] [ 57%] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-depends_on1-True-True] test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-None-False-False] test_parallel_replicas_increase_error_count/test.py::test_increase_error_count [gw2] [ 58%] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-None-False-False] test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-None-False-True] [gw4] [ 59%] PASSED test_polymorphic_parts/test.py::test_polymorphic_parts_basics[first_node0-second_node0] test_polymorphic_parts/test.py::test_polymorphic_parts_basics[first_node1-second_node1] [gw0] [ 60%] PASSED test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-11-10000-SELECT sum(key) FROM {table_name}] test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-11-100000-SELECT sum(key) FROM {table_name}] [gw2] [ 61%] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-None-False-True] test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-None-True-False] test_match_process_uid_against_data_owner/test.py::test_different_user [gw2] [ 62%] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-None-True-False] test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-None-True-True] [gw0] [ 63%] PASSED test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-11-100000-SELECT sum(key) FROM {table_name}] test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-101-1000-SELECT sum(key) FROM {table_name}] [gw2] [ 64%] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-None-True-True] test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-depends_on1-False-False] [gw5] [ 65%] PASSED test_keeper_snapshot_small_distance/test.py::test_snapshot_and_load test_log_levels_update/test.py::test_log_levels_update [gw2] [ 66%] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-depends_on1-False-False] [gw6] [ 67%] PASSED test_parallel_replicas_increase_error_count/test.py::test_increase_error_count test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-depends_on1-False-True] [gw0] [ 68%] PASSED test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-101-1000-SELECT sum(key) FROM {table_name}] test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-101-10000-SELECT sum(key) FROM {table_name}] [gw9] [ 69%] PASSED test_match_process_uid_against_data_owner/test.py::test_different_user [gw2] [ 70%] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-depends_on1-False-True] test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-depends_on1-True-False] [gw4] [ 71%] PASSED test_polymorphic_parts/test.py::test_polymorphic_parts_basics[first_node1-second_node1] test_polymorphic_parts/test.py::test_polymorphic_parts_index [gw4] [ 72%] PASSED test_polymorphic_parts/test.py::test_polymorphic_parts_index test_polymorphic_parts/test.py::test_polymorphic_parts_non_adaptive [gw0] [ 73%] PASSED test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-101-10000-SELECT sum(key) FROM {table_name}] test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-101-100000-SELECT sum(key) FROM {table_name}] [gw2] [ 74%] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-depends_on1-True-False] test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-depends_on1-True-True] [gw4] [ 75%] PASSED test_polymorphic_parts/test.py::test_polymorphic_parts_non_adaptive [gw5] [ 76%] PASSED test_log_levels_update/test.py::test_log_levels_update test_merge_tree_check_part_with_cache/test.py::test_check_part_with_cache [gw2] [ 77%] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-depends_on1-True-True] test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-None-False-False] [gw0] [ 78%] PASSED test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-101-100000-SELECT sum(key) FROM {table_name}] test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-11-1000-SELECT sum(key) FROM {table_name}] [gw2] [ 79%] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-None-False-False] test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-None-False-True] [gw2] [ 80%] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-None-False-True] test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-None-True-False] [gw5] [ 81%] SKIPPED test_merge_tree_check_part_with_cache/test.py::test_check_part_with_cache [gw0] [ 82%] PASSED test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-11-1000-SELECT sum(key) FROM {table_name}] test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-11-10000-SELECT sum(key) FROM {table_name}] [gw2] [ 83%] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-None-True-False] test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-None-True-True] [gw2] [ 84%] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-None-True-True] test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-depends_on1-False-False] [gw0] [ 85%] PASSED test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-11-10000-SELECT sum(key) FROM {table_name}] test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-11-100000-SELECT sum(key) FROM {table_name}] [gw2] [ 86%] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-depends_on1-False-False] test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-depends_on1-False-True] [gw2] [ 87%] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-depends_on1-False-True] [gw0] [ 88%] PASSED test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-11-100000-SELECT sum(key) FROM {table_name}] test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-depends_on1-True-False] [gw2] [ 89%] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-depends_on1-True-False] test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-depends_on1-True-True] [gw2] [ 90%] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-depends_on1-True-True] test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-None-False-False] [gw2] [ 91%] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-None-False-False] test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-None-False-True] [gw2] [ 92%] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-None-False-True] test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-None-True-False] [gw2] [ 93%] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-None-True-False] test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-None-True-True] [gw2] [ 94%] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-None-True-True] test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-depends_on1-False-False] [gw2] [ 95%] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-depends_on1-False-False] test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-depends_on1-False-True] [gw2] [ 96%] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-depends_on1-False-True] test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-depends_on1-True-False] [gw2] [ 97%] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-depends_on1-True-False] test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-depends_on1-True-True] [gw2] [ 98%] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-depends_on1-True-True] [gw8] [ 99%] PASSED test_log_query_probability/test.py::test_log_quries_probability_two test_grant_and_revoke/test_without_table_engine_grant.py::test_table_engine_and_source_grant [gw8] [100%] PASSED test_grant_and_revoke/test_without_table_engine_grant.py::test_table_engine_and_source_grant =================================== FAILURES =================================== ______________________ test_reconfig_remove_2_and_leader _______________________ [gw6] linux -- Python 3.10.12 /usr/bin/python3 started_cluster = def test_reconfig_remove_2_and_leader(started_cluster): """ Remove 2 followers from a cluster of 5. Remove leader from 3 nodes. """ global zk1, zk2, zk3, zk4, zk5 zk1 = create_client(node1) config = ku.get_config_str(zk1) print("Initial config", config) assert len(config.split("\n")) == 5 for i in range(100): zk1.create(f"/test_two_{i}", "somedata") zk4 = create_client(node4) zk4.sync("/test_two_0") ku.wait_configs_equal(config, zk4) zk5 = create_client(node5) zk5.sync("/test_two_0") ku.wait_configs_equal(config, zk5) for i in range(100): assert zk4.exists(f"test_two_{i}") assert zk5.exists(f"test_two_{i}") zk4.create(f"/test_two_{100 + i}", "otherdata") zk2 = create_client(node2) config = zk2.reconfig(joining=None, leaving="4,5", new_members=None) print("After removing 4,5", config) assert len(config.split("\n")) == 3 assert "node1" in config assert "node2" in config assert "node3" in config assert "node4" not in config assert "node5" not in config zk1.stop() zk1 = create_client(node1) zk1.sync("/test_two_0") ku.wait_configs_equal(config, zk1) for i in range(200): assert zk1.exists(f"test_two_{i}") assert zk2.exists(f"test_two_{i}") assert not node1.contains_in_log(log_msg_removed) assert not node2.contains_in_log(log_msg_removed) assert not node3.contains_in_log(log_msg_removed) assert node4.contains_in_log(log_msg_removed) assert node5.contains_in_log(log_msg_removed) assert ku.is_leader(cluster, node1) for i in range(100): > zk1.create(f"/test_leader_{i}", "somedata") test_keeper_reconfig_remove_many/test.py:103: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/keeper_utils.py:175: in create self.execute_query(f"create '{path}' '{value}'", timeout) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = query = "create '/test_leader_79' 'somedata'", timeout = 60.0 def execute_query(self, query: str, timeout: float = 60.0) -> str: output = io.BytesIO() self.proc.stdin.write(query.encode() + b"\n") self.proc.stdin.flush() events = self.poller.poll(timeout) if not events: raise TimeoutError(f"Keeper client returned no output") for fd_num, event in events: if event & (select.EPOLLIN | select.EPOLLPRI): file = self._fd_nums[fd_num] if file == self.proc.stdout: while True: chunk = file.readline() if chunk.endswith(self.SEPARATOR): break output.write(chunk) elif file == self.proc.stderr: self.proc.stdout.readline() > raise KeeperException(self.proc.stderr.readline().strip().decode()) E helpers.keeper_utils.KeeperException: Coordination error: Operation timeout, path /test_leader_79 helpers/keeper_utils.py:160: KeeperException ---------------------------- Captured stdout setup ----------------------------- Copy common default production configuration from /clickhouse-config. Files: config.xml, users.xml Copy common default production configuration from /clickhouse-config. Files: config.xml, users.xml Copy common default production configuration from /clickhouse-config. Files: config.xml, users.xml Copy common default production configuration from /clickhouse-config. Files: config.xml, users.xml Copy common default production configuration from /clickhouse-config. Files: config.xml, users.xml ------------------------------ Captured log setup ------------------------------ 2025-11-13 09:08:59.426000 [ 588 ] INFO : Running tests in /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/test.py (cluster.py:2738, start) 2025-11-13 09:08:59.427000 [ 588 ] DEBUG : Cluster start called. is_up=False (cluster.py:2745, start) 2025-11-13 09:08:59.486000 [ 588 ] DEBUG : Docker networks for project roottestkeeperreconfigremovemany-gw6 are NETWORK ID NAME DRIVER SCOPE (cluster.py:830, print_all_docker_pieces) 2025-11-13 09:08:59.559000 [ 588 ] DEBUG : Docker containers for project roottestkeeperreconfigremovemany-gw6 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES (cluster.py:838, print_all_docker_pieces) 2025-11-13 09:08:59.740000 [ 588 ] DEBUG : Docker volumes for project roottestkeeperreconfigremovemany-gw6 are DRIVER VOLUME NAME (cluster.py:846, print_all_docker_pieces) 2025-11-13 09:08:59.740000 [ 588 ] DEBUG : Cleanup called (cluster.py:851, cleanup) 2025-11-13 09:08:59.775000 [ 588 ] DEBUG : Docker networks for project roottestkeeperreconfigremovemany-gw6 are NETWORK ID NAME DRIVER SCOPE (cluster.py:830, print_all_docker_pieces) 2025-11-13 09:08:59.879000 [ 588 ] DEBUG : Docker containers for project roottestkeeperreconfigremovemany-gw6 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES (cluster.py:838, print_all_docker_pieces) 2025-11-13 09:08:59.989000 [ 588 ] DEBUG : Docker volumes for project roottestkeeperreconfigremovemany-gw6 are DRIVER VOLUME NAME (cluster.py:846, print_all_docker_pieces) 2025-11-13 09:08:59.990000 [ 588 ] DEBUG : Command:[docker container list --all --filter name='^/roottestkeeperreconfigremovemany-gw6-.*-1$' --format '{{.ID}}:{{.Names}}'] (cluster.py:121, run_and_check) 2025-11-13 09:09:00.113000 [ 588 ] DEBUG : Unstopped containers: {} (cluster.py:865, cleanup) 2025-11-13 09:09:00.113000 [ 588 ] DEBUG : No running containers for project: roottestkeeperreconfigremovemany-gw6 (cluster.py:879, cleanup) 2025-11-13 09:09:00.113000 [ 588 ] DEBUG : Trying to prune unused networks... (cluster.py:885, cleanup) 2025-11-13 09:09:00.138000 [ 588 ] DEBUG : Trying to prune unused images... (cluster.py:901, cleanup) 2025-11-13 09:09:00.138000 [ 588 ] DEBUG : Command:[docker image prune -f] (cluster.py:121, run_and_check) 2025-11-13 09:09:00.237000 [ 588 ] DEBUG : Stdout:Total reclaimed space: 0B (cluster.py:145, run_and_check) 2025-11-13 09:09:00.237000 [ 588 ] DEBUG : Images pruned (cluster.py:904, cleanup) 2025-11-13 09:09:00.237000 [ 588 ] DEBUG : Trying to prune unused volumes... (cluster.py:910, cleanup) 2025-11-13 09:09:00.237000 [ 588 ] DEBUG : Command:[docker volume ls | wc -l] (cluster.py:121, run_and_check) 2025-11-13 09:09:00.284000 [ 588 ] DEBUG : Stdout:5 (cluster.py:145, run_and_check) 2025-11-13 09:09:00.297000 [ 588 ] DEBUG : Command:[docker volume prune -f] (cluster.py:121, run_and_check) 2025-11-13 09:09:00.361000 [ 588 ] DEBUG : Stdout:Total reclaimed space: 0B (cluster.py:145, run_and_check) 2025-11-13 09:09:00.361000 [ 588 ] DEBUG : Volumes pruned: 5 (cluster.py:915, cleanup) 2025-11-13 09:09:00.361000 [ 588 ] DEBUG : Setup directory for instance: node1 (cluster.py:2758, start) 2025-11-13 09:09:00.362000 [ 588 ] DEBUG : Create directory for configuration generated in this helper (cluster.py:4628, create_dir) 2025-11-13 09:09:00.363000 [ 588 ] DEBUG : Create directory for common tests configuration (cluster.py:4633, create_dir) 2025-11-13 09:09:00.363000 [ 588 ] DEBUG : Copy common configuration from helpers (cluster.py:4653, create_dir) 2025-11-13 09:09:00.365000 [ 588 ] DEBUG : Generate and write macros file (cluster.py:4705, create_dir) 2025-11-13 09:09:00.368000 [ 588 ] DEBUG : Copy custom test config files ['/ClickHouse/tests/integration/test_keeper_reconfig_remove_many/configs/keeper1.xml'] to /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node1/configs/config.d (cluster.py:4741, create_dir) 2025-11-13 09:09:00.370000 [ 588 ] DEBUG : Setup database dir /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node1/database (cluster.py:4758, create_dir) 2025-11-13 09:09:00.370000 [ 588 ] DEBUG : Setup logs dir /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node1/logs (cluster.py:4769, create_dir) 2025-11-13 09:09:00.371000 [ 588 ] DEBUG : Entrypoint cmd: ["clickhouse", "server", "--config-file=/etc/clickhouse-server/config.xml", "--log-file=/var/log/clickhouse-server/clickhouse-server.log", "--errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log", "--"] (cluster.py:4850, create_dir) 2025-11-13 09:09:00.371000 [ 588 ] DEBUG : Setup directory for instance: node2 (cluster.py:2758, start) 2025-11-13 09:09:00.372000 [ 588 ] DEBUG : Create directory for configuration generated in this helper (cluster.py:4628, create_dir) 2025-11-13 09:09:00.372000 [ 588 ] DEBUG : Create directory for common tests configuration (cluster.py:4633, create_dir) 2025-11-13 09:09:00.372000 [ 588 ] DEBUG : Copy common configuration from helpers (cluster.py:4653, create_dir) 2025-11-13 09:09:00.373000 [ 588 ] DEBUG : Generate and write macros file (cluster.py:4705, create_dir) 2025-11-13 09:09:00.374000 [ 588 ] DEBUG : Copy custom test config files ['/ClickHouse/tests/integration/test_keeper_reconfig_remove_many/configs/keeper2.xml'] to /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node2/configs/config.d (cluster.py:4741, create_dir) 2025-11-13 09:09:00.375000 [ 588 ] DEBUG : Setup database dir /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node2/database (cluster.py:4758, create_dir) 2025-11-13 09:09:00.375000 [ 588 ] DEBUG : Setup logs dir /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node2/logs (cluster.py:4769, create_dir) 2025-11-13 09:09:00.375000 [ 588 ] DEBUG : Entrypoint cmd: ["clickhouse", "server", "--config-file=/etc/clickhouse-server/config.xml", "--log-file=/var/log/clickhouse-server/clickhouse-server.log", "--errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log", "--"] (cluster.py:4850, create_dir) 2025-11-13 09:09:00.375000 [ 588 ] DEBUG : Setup directory for instance: node3 (cluster.py:2758, start) 2025-11-13 09:09:00.378000 [ 588 ] DEBUG : Create directory for configuration generated in this helper (cluster.py:4628, create_dir) 2025-11-13 09:09:00.378000 [ 588 ] DEBUG : Create directory for common tests configuration (cluster.py:4633, create_dir) 2025-11-13 09:09:00.378000 [ 588 ] DEBUG : Copy common configuration from helpers (cluster.py:4653, create_dir) 2025-11-13 09:09:00.379000 [ 588 ] DEBUG : Generate and write macros file (cluster.py:4705, create_dir) 2025-11-13 09:09:00.379000 [ 588 ] DEBUG : Copy custom test config files ['/ClickHouse/tests/integration/test_keeper_reconfig_remove_many/configs/keeper3.xml'] to /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node3/configs/config.d (cluster.py:4741, create_dir) 2025-11-13 09:09:00.380000 [ 588 ] DEBUG : Setup database dir /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node3/database (cluster.py:4758, create_dir) 2025-11-13 09:09:00.380000 [ 588 ] DEBUG : Setup logs dir /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node3/logs (cluster.py:4769, create_dir) 2025-11-13 09:09:00.380000 [ 588 ] DEBUG : Entrypoint cmd: ["clickhouse", "server", "--config-file=/etc/clickhouse-server/config.xml", "--log-file=/var/log/clickhouse-server/clickhouse-server.log", "--errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log", "--"] (cluster.py:4850, create_dir) 2025-11-13 09:09:00.380000 [ 588 ] DEBUG : Setup directory for instance: node4 (cluster.py:2758, start) 2025-11-13 09:09:00.381000 [ 588 ] DEBUG : Create directory for configuration generated in this helper (cluster.py:4628, create_dir) 2025-11-13 09:09:00.381000 [ 588 ] DEBUG : Create directory for common tests configuration (cluster.py:4633, create_dir) 2025-11-13 09:09:00.381000 [ 588 ] DEBUG : Copy common configuration from helpers (cluster.py:4653, create_dir) 2025-11-13 09:09:00.382000 [ 588 ] DEBUG : Generate and write macros file (cluster.py:4705, create_dir) 2025-11-13 09:09:00.382000 [ 588 ] DEBUG : Copy custom test config files ['/ClickHouse/tests/integration/test_keeper_reconfig_remove_many/configs/keeper4.xml'] to /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node4/configs/config.d (cluster.py:4741, create_dir) 2025-11-13 09:09:00.383000 [ 588 ] DEBUG : Setup database dir /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node4/database (cluster.py:4758, create_dir) 2025-11-13 09:09:00.383000 [ 588 ] DEBUG : Setup logs dir /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node4/logs (cluster.py:4769, create_dir) 2025-11-13 09:09:00.383000 [ 588 ] DEBUG : Entrypoint cmd: ["clickhouse", "server", "--config-file=/etc/clickhouse-server/config.xml", "--log-file=/var/log/clickhouse-server/clickhouse-server.log", "--errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log", "--"] (cluster.py:4850, create_dir) 2025-11-13 09:09:00.383000 [ 588 ] DEBUG : Setup directory for instance: node5 (cluster.py:2758, start) 2025-11-13 09:09:00.384000 [ 588 ] DEBUG : Create directory for configuration generated in this helper (cluster.py:4628, create_dir) 2025-11-13 09:09:00.384000 [ 588 ] DEBUG : Create directory for common tests configuration (cluster.py:4633, create_dir) 2025-11-13 09:09:00.384000 [ 588 ] DEBUG : Copy common configuration from helpers (cluster.py:4653, create_dir) 2025-11-13 09:09:00.385000 [ 588 ] DEBUG : Generate and write macros file (cluster.py:4705, create_dir) 2025-11-13 09:09:00.386000 [ 588 ] DEBUG : Copy custom test config files ['/ClickHouse/tests/integration/test_keeper_reconfig_remove_many/configs/keeper5.xml'] to /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node5/configs/config.d (cluster.py:4741, create_dir) 2025-11-13 09:09:00.388000 [ 588 ] DEBUG : Setup database dir /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node5/database (cluster.py:4758, create_dir) 2025-11-13 09:09:00.388000 [ 588 ] DEBUG : Setup logs dir /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node5/logs (cluster.py:4769, create_dir) 2025-11-13 09:09:00.388000 [ 588 ] DEBUG : Entrypoint cmd: ["clickhouse", "server", "--config-file=/etc/clickhouse-server/config.xml", "--log-file=/var/log/clickhouse-server/clickhouse-server.log", "--errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log", "--"] (cluster.py:4850, create_dir) 2025-11-13 09:09:00.388000 [ 588 ] DEBUG : Env {'ASAN_OPTIONS': 'use_sigaltstack=0', 'TSAN_OPTIONS': 'use_sigaltstack=0', 'CLICKHOUSE_WATCHDOG_ENABLE': '0', 'CLICKHOUSE_NATS_TLS_SECURE': '0', 'LLVM_PROFILE_FILE': '/var/lib/clickhouse/server_%h_%p_%m.profraw'} stored in /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/.env (cluster.py:96, _create_env_file) 2025-11-13 09:09:00.392000 [ 588 ] DEBUG : Trying paths: ['/root/.docker/config.json', '/root/.dockercfg'] (config.py:21, find_config_file) 2025-11-13 09:09:00.392000 [ 588 ] DEBUG : No config file found (config.py:28, find_config_file) 2025-11-13 09:09:00.394000 [ 588 ] DEBUG : Trying paths: ['/root/.docker/config.json', '/root/.dockercfg'] (config.py:21, find_config_file) 2025-11-13 09:09:00.394000 [ 588 ] DEBUG : No config file found (config.py:28, find_config_file) 2025-11-13 09:09:00.440000 [ 588 ] DEBUG : http://localhost:None "GET /version HTTP/1.1" 200 826 (connectionpool.py:547, _make_request) 2025-11-13 09:09:00.441000 [ 588 ] DEBUG : Command:[docker compose --env-file /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/.env --project-name roottestkeeperreconfigremovemany-gw6 --file /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node1/docker-compose.yml --file /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node2/docker-compose.yml --file /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node3/docker-compose.yml --file /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node4/docker-compose.yml --file /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node5/docker-compose.yml pull] (cluster.py:121, run_and_check) 2025-11-13 09:09:11.308000 [ 588 ] DEBUG : Stderr: node1 Skipped - Image is already being pulled by node5 (cluster.py:147, run_and_check) 2025-11-13 09:09:11.315000 [ 588 ] DEBUG : Stderr: node2 Skipped - Image is already being pulled by node5 (cluster.py:147, run_and_check) 2025-11-13 09:09:11.315000 [ 588 ] DEBUG : Stderr: node3 Skipped - Image is already being pulled by node5 (cluster.py:147, run_and_check) 2025-11-13 09:09:11.315000 [ 588 ] DEBUG : Stderr: node4 Skipped - Image is already being pulled by node5 (cluster.py:147, run_and_check) 2025-11-13 09:09:11.315000 [ 588 ] DEBUG : Stderr: node5 Pulling (cluster.py:147, run_and_check) 2025-11-13 09:09:11.315000 [ 588 ] DEBUG : Stderr: node5 Pulled (cluster.py:147, run_and_check) 2025-11-13 09:09:11.316000 [ 588 ] DEBUG : ('Trying to create ClickHouse instance by command %s', 'docker compose --env-file /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/.env --project-name roottestkeeperreconfigremovemany-gw6 --file /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node1/docker-compose.yml --file /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node2/docker-compose.yml --file /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node3/docker-compose.yml --file /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node4/docker-compose.yml --file /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node5/docker-compose.yml up -d --no-recreate') (cluster.py:3139, start) 2025-11-13 09:09:11.316000 [ 588 ] DEBUG : Command:[docker compose --env-file /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/.env --project-name roottestkeeperreconfigremovemany-gw6 --file /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node1/docker-compose.yml --file /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node2/docker-compose.yml --file /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node3/docker-compose.yml --file /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node4/docker-compose.yml --file /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node5/docker-compose.yml up -d --no-recreate] (cluster.py:121, run_and_check) 2025-11-13 09:09:15.229000 [ 588 ] DEBUG : Stderr: Network roottestkeeperreconfigremovemany-gw6_default Creating (cluster.py:147, run_and_check) 2025-11-13 09:09:15.229000 [ 588 ] DEBUG : Stderr: Network roottestkeeperreconfigremovemany-gw6_default Created (cluster.py:147, run_and_check) 2025-11-13 09:09:15.229000 [ 588 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node3-1 Creating (cluster.py:147, run_and_check) 2025-11-13 09:09:15.230000 [ 588 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node4-1 Creating (cluster.py:147, run_and_check) 2025-11-13 09:09:15.230000 [ 588 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node5-1 Creating (cluster.py:147, run_and_check) 2025-11-13 09:09:15.230000 [ 588 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node1-1 Creating (cluster.py:147, run_and_check) 2025-11-13 09:09:15.230000 [ 588 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node2-1 Creating (cluster.py:147, run_and_check) 2025-11-13 09:09:15.230000 [ 588 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node2-1 Created (cluster.py:147, run_and_check) 2025-11-13 09:09:15.230000 [ 588 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node5-1 Created (cluster.py:147, run_and_check) 2025-11-13 09:09:15.230000 [ 588 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node1-1 Created (cluster.py:147, run_and_check) 2025-11-13 09:09:15.230000 [ 588 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node3-1 Created (cluster.py:147, run_and_check) 2025-11-13 09:09:15.230000 [ 588 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node4-1 Created (cluster.py:147, run_and_check) 2025-11-13 09:09:15.230000 [ 588 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node4-1 Starting (cluster.py:147, run_and_check) 2025-11-13 09:09:15.230000 [ 588 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node5-1 Starting (cluster.py:147, run_and_check) 2025-11-13 09:09:15.231000 [ 588 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node1-1 Starting (cluster.py:147, run_and_check) 2025-11-13 09:09:15.231000 [ 588 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node2-1 Starting (cluster.py:147, run_and_check) 2025-11-13 09:09:15.231000 [ 588 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node3-1 Starting (cluster.py:147, run_and_check) 2025-11-13 09:09:15.231000 [ 588 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node1-1 Started (cluster.py:147, run_and_check) 2025-11-13 09:09:15.231000 [ 588 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node4-1 Started (cluster.py:147, run_and_check) 2025-11-13 09:09:15.231000 [ 588 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node5-1 Started (cluster.py:147, run_and_check) 2025-11-13 09:09:15.231000 [ 588 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node2-1 Started (cluster.py:147, run_and_check) 2025-11-13 09:09:15.231000 [ 588 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node3-1 Started (cluster.py:147, run_and_check) 2025-11-13 09:09:15.231000 [ 588 ] DEBUG : ClickHouse instance created (cluster.py:3147, start) 2025-11-13 09:09:15.231000 [ 588 ] DEBUG : get_instance_ip instance_name=node1 (cluster.py:2005, get_instance_ip) 2025-11-13 09:09:15.240000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestkeeperreconfigremovemany-gw6-node1-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:15.242000 [ 588 ] DEBUG : get_instance_ip instance_name=node1 (cluster.py:2015, get_instance_global_ipv6) 2025-11-13 09:09:15.283000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestkeeperreconfigremovemany-gw6-node1-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:15.285000 [ 588 ] DEBUG : Waiting for ClickHouse start in node1, ip: 172.16.5.6... (cluster.py:3155, start) 2025-11-13 09:09:15.290000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestkeeperreconfigremovemany-gw6-node1-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:15.341000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/a245d1133b0984828ce39ed27b9fe6f57c052431b7af3b09db1d6754c7bfcf22/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:15.451000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/a245d1133b0984828ce39ed27b9fe6f57c052431b7af3b09db1d6754c7bfcf22/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:15.555000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/a245d1133b0984828ce39ed27b9fe6f57c052431b7af3b09db1d6754c7bfcf22/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:15.665000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/a245d1133b0984828ce39ed27b9fe6f57c052431b7af3b09db1d6754c7bfcf22/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:15.776000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/a245d1133b0984828ce39ed27b9fe6f57c052431b7af3b09db1d6754c7bfcf22/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:15.887000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/a245d1133b0984828ce39ed27b9fe6f57c052431b7af3b09db1d6754c7bfcf22/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:15.993000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/a245d1133b0984828ce39ed27b9fe6f57c052431b7af3b09db1d6754c7bfcf22/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:16.099000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/a245d1133b0984828ce39ed27b9fe6f57c052431b7af3b09db1d6754c7bfcf22/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:16.206000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/a245d1133b0984828ce39ed27b9fe6f57c052431b7af3b09db1d6754c7bfcf22/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:16.324000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/a245d1133b0984828ce39ed27b9fe6f57c052431b7af3b09db1d6754c7bfcf22/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:16.459000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/a245d1133b0984828ce39ed27b9fe6f57c052431b7af3b09db1d6754c7bfcf22/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:16.571000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/a245d1133b0984828ce39ed27b9fe6f57c052431b7af3b09db1d6754c7bfcf22/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:16.677000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/a245d1133b0984828ce39ed27b9fe6f57c052431b7af3b09db1d6754c7bfcf22/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:16.782000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/a245d1133b0984828ce39ed27b9fe6f57c052431b7af3b09db1d6754c7bfcf22/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:16.887000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/a245d1133b0984828ce39ed27b9fe6f57c052431b7af3b09db1d6754c7bfcf22/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:16.996000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/a245d1133b0984828ce39ed27b9fe6f57c052431b7af3b09db1d6754c7bfcf22/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:17.106000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/a245d1133b0984828ce39ed27b9fe6f57c052431b7af3b09db1d6754c7bfcf22/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:17.212000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/a245d1133b0984828ce39ed27b9fe6f57c052431b7af3b09db1d6754c7bfcf22/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:17.320000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/a245d1133b0984828ce39ed27b9fe6f57c052431b7af3b09db1d6754c7bfcf22/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:17.425000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/a245d1133b0984828ce39ed27b9fe6f57c052431b7af3b09db1d6754c7bfcf22/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:17.529000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/a245d1133b0984828ce39ed27b9fe6f57c052431b7af3b09db1d6754c7bfcf22/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:17.637000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/a245d1133b0984828ce39ed27b9fe6f57c052431b7af3b09db1d6754c7bfcf22/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:17.742000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/a245d1133b0984828ce39ed27b9fe6f57c052431b7af3b09db1d6754c7bfcf22/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:17.855000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/a245d1133b0984828ce39ed27b9fe6f57c052431b7af3b09db1d6754c7bfcf22/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:17.971000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/a245d1133b0984828ce39ed27b9fe6f57c052431b7af3b09db1d6754c7bfcf22/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:18.077000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/a245d1133b0984828ce39ed27b9fe6f57c052431b7af3b09db1d6754c7bfcf22/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:18.206000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/a245d1133b0984828ce39ed27b9fe6f57c052431b7af3b09db1d6754c7bfcf22/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:18.313000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/a245d1133b0984828ce39ed27b9fe6f57c052431b7af3b09db1d6754c7bfcf22/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:18.424000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/a245d1133b0984828ce39ed27b9fe6f57c052431b7af3b09db1d6754c7bfcf22/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:18.529000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/a245d1133b0984828ce39ed27b9fe6f57c052431b7af3b09db1d6754c7bfcf22/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:18.635000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/a245d1133b0984828ce39ed27b9fe6f57c052431b7af3b09db1d6754c7bfcf22/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:18.746000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/a245d1133b0984828ce39ed27b9fe6f57c052431b7af3b09db1d6754c7bfcf22/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:18.854000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/a245d1133b0984828ce39ed27b9fe6f57c052431b7af3b09db1d6754c7bfcf22/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:18.985000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/a245d1133b0984828ce39ed27b9fe6f57c052431b7af3b09db1d6754c7bfcf22/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:19.105000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/a245d1133b0984828ce39ed27b9fe6f57c052431b7af3b09db1d6754c7bfcf22/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:19.210000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/a245d1133b0984828ce39ed27b9fe6f57c052431b7af3b09db1d6754c7bfcf22/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:19.317000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/a245d1133b0984828ce39ed27b9fe6f57c052431b7af3b09db1d6754c7bfcf22/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:19.464000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/a245d1133b0984828ce39ed27b9fe6f57c052431b7af3b09db1d6754c7bfcf22/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:19.575000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/a245d1133b0984828ce39ed27b9fe6f57c052431b7af3b09db1d6754c7bfcf22/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:19.680000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/a245d1133b0984828ce39ed27b9fe6f57c052431b7af3b09db1d6754c7bfcf22/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:19.786000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/a245d1133b0984828ce39ed27b9fe6f57c052431b7af3b09db1d6754c7bfcf22/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:19.896000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/a245d1133b0984828ce39ed27b9fe6f57c052431b7af3b09db1d6754c7bfcf22/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:20.013000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/a245d1133b0984828ce39ed27b9fe6f57c052431b7af3b09db1d6754c7bfcf22/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:20.122000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/a245d1133b0984828ce39ed27b9fe6f57c052431b7af3b09db1d6754c7bfcf22/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:20.231000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/a245d1133b0984828ce39ed27b9fe6f57c052431b7af3b09db1d6754c7bfcf22/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:20.341000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/a245d1133b0984828ce39ed27b9fe6f57c052431b7af3b09db1d6754c7bfcf22/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:20.452000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/a245d1133b0984828ce39ed27b9fe6f57c052431b7af3b09db1d6754c7bfcf22/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:20.557000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/a245d1133b0984828ce39ed27b9fe6f57c052431b7af3b09db1d6754c7bfcf22/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:20.663000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/a245d1133b0984828ce39ed27b9fe6f57c052431b7af3b09db1d6754c7bfcf22/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:20.769000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/a245d1133b0984828ce39ed27b9fe6f57c052431b7af3b09db1d6754c7bfcf22/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:20.873000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/a245d1133b0984828ce39ed27b9fe6f57c052431b7af3b09db1d6754c7bfcf22/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:20.978000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/a245d1133b0984828ce39ed27b9fe6f57c052431b7af3b09db1d6754c7bfcf22/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:21.088000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/a245d1133b0984828ce39ed27b9fe6f57c052431b7af3b09db1d6754c7bfcf22/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:21.224000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/a245d1133b0984828ce39ed27b9fe6f57c052431b7af3b09db1d6754c7bfcf22/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:21.335000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/a245d1133b0984828ce39ed27b9fe6f57c052431b7af3b09db1d6754c7bfcf22/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:21.446000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/a245d1133b0984828ce39ed27b9fe6f57c052431b7af3b09db1d6754c7bfcf22/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:21.552000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/a245d1133b0984828ce39ed27b9fe6f57c052431b7af3b09db1d6754c7bfcf22/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:21.658000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/a245d1133b0984828ce39ed27b9fe6f57c052431b7af3b09db1d6754c7bfcf22/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:21.797000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/a245d1133b0984828ce39ed27b9fe6f57c052431b7af3b09db1d6754c7bfcf22/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:21.905000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/a245d1133b0984828ce39ed27b9fe6f57c052431b7af3b09db1d6754c7bfcf22/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:22.010000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/a245d1133b0984828ce39ed27b9fe6f57c052431b7af3b09db1d6754c7bfcf22/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:22.116000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/a245d1133b0984828ce39ed27b9fe6f57c052431b7af3b09db1d6754c7bfcf22/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:22.223000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/a245d1133b0984828ce39ed27b9fe6f57c052431b7af3b09db1d6754c7bfcf22/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:22.334000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/a245d1133b0984828ce39ed27b9fe6f57c052431b7af3b09db1d6754c7bfcf22/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:22.477000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/a245d1133b0984828ce39ed27b9fe6f57c052431b7af3b09db1d6754c7bfcf22/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:22.583000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/a245d1133b0984828ce39ed27b9fe6f57c052431b7af3b09db1d6754c7bfcf22/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:22.690000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/a245d1133b0984828ce39ed27b9fe6f57c052431b7af3b09db1d6754c7bfcf22/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:22.797000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/a245d1133b0984828ce39ed27b9fe6f57c052431b7af3b09db1d6754c7bfcf22/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:22.906000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/a245d1133b0984828ce39ed27b9fe6f57c052431b7af3b09db1d6754c7bfcf22/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:23.016000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/a245d1133b0984828ce39ed27b9fe6f57c052431b7af3b09db1d6754c7bfcf22/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:23.123000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/a245d1133b0984828ce39ed27b9fe6f57c052431b7af3b09db1d6754c7bfcf22/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:23.231000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/a245d1133b0984828ce39ed27b9fe6f57c052431b7af3b09db1d6754c7bfcf22/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:23.345000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/a245d1133b0984828ce39ed27b9fe6f57c052431b7af3b09db1d6754c7bfcf22/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:23.449000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/a245d1133b0984828ce39ed27b9fe6f57c052431b7af3b09db1d6754c7bfcf22/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:23.565000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/a245d1133b0984828ce39ed27b9fe6f57c052431b7af3b09db1d6754c7bfcf22/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:23.670000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/a245d1133b0984828ce39ed27b9fe6f57c052431b7af3b09db1d6754c7bfcf22/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:23.780000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/a245d1133b0984828ce39ed27b9fe6f57c052431b7af3b09db1d6754c7bfcf22/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:23.886000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/a245d1133b0984828ce39ed27b9fe6f57c052431b7af3b09db1d6754c7bfcf22/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:23.990000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/a245d1133b0984828ce39ed27b9fe6f57c052431b7af3b09db1d6754c7bfcf22/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:24.098000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/a245d1133b0984828ce39ed27b9fe6f57c052431b7af3b09db1d6754c7bfcf22/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:24.206000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/a245d1133b0984828ce39ed27b9fe6f57c052431b7af3b09db1d6754c7bfcf22/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:24.315000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/a245d1133b0984828ce39ed27b9fe6f57c052431b7af3b09db1d6754c7bfcf22/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:24.421000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/a245d1133b0984828ce39ed27b9fe6f57c052431b7af3b09db1d6754c7bfcf22/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:24.529000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/a245d1133b0984828ce39ed27b9fe6f57c052431b7af3b09db1d6754c7bfcf22/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:24.638000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/a245d1133b0984828ce39ed27b9fe6f57c052431b7af3b09db1d6754c7bfcf22/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:24.746000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/a245d1133b0984828ce39ed27b9fe6f57c052431b7af3b09db1d6754c7bfcf22/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:24.855000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/a245d1133b0984828ce39ed27b9fe6f57c052431b7af3b09db1d6754c7bfcf22/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:24.964000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/a245d1133b0984828ce39ed27b9fe6f57c052431b7af3b09db1d6754c7bfcf22/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:25.069000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/a245d1133b0984828ce39ed27b9fe6f57c052431b7af3b09db1d6754c7bfcf22/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:25.176000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/a245d1133b0984828ce39ed27b9fe6f57c052431b7af3b09db1d6754c7bfcf22/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:25.281000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/a245d1133b0984828ce39ed27b9fe6f57c052431b7af3b09db1d6754c7bfcf22/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:25.394000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/a245d1133b0984828ce39ed27b9fe6f57c052431b7af3b09db1d6754c7bfcf22/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:25.498000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/a245d1133b0984828ce39ed27b9fe6f57c052431b7af3b09db1d6754c7bfcf22/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:25.603000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/a245d1133b0984828ce39ed27b9fe6f57c052431b7af3b09db1d6754c7bfcf22/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:25.707000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/a245d1133b0984828ce39ed27b9fe6f57c052431b7af3b09db1d6754c7bfcf22/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:25.813000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/a245d1133b0984828ce39ed27b9fe6f57c052431b7af3b09db1d6754c7bfcf22/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:25.923000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/a245d1133b0984828ce39ed27b9fe6f57c052431b7af3b09db1d6754c7bfcf22/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:25.925000 [ 588 ] DEBUG : ClickHouse node1 started (cluster.py:3159, start) 2025-11-13 09:09:25.926000 [ 588 ] DEBUG : get_instance_ip instance_name=node2 (cluster.py:2005, get_instance_ip) 2025-11-13 09:09:25.935000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestkeeperreconfigremovemany-gw6-node2-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:25.936000 [ 588 ] DEBUG : get_instance_ip instance_name=node2 (cluster.py:2015, get_instance_global_ipv6) 2025-11-13 09:09:25.939000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestkeeperreconfigremovemany-gw6-node2-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:25.939000 [ 588 ] DEBUG : Waiting for ClickHouse start in node2, ip: 172.16.5.4... (cluster.py:3155, start) 2025-11-13 09:09:25.943000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestkeeperreconfigremovemany-gw6-node2-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:25.948000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/33e560b130d782b733ad2050802881a70fb40c10a1027591bd5cc44a83e0ab50/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:26.055000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/33e560b130d782b733ad2050802881a70fb40c10a1027591bd5cc44a83e0ab50/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:26.165000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/33e560b130d782b733ad2050802881a70fb40c10a1027591bd5cc44a83e0ab50/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:26.271000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/33e560b130d782b733ad2050802881a70fb40c10a1027591bd5cc44a83e0ab50/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:26.375000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/33e560b130d782b733ad2050802881a70fb40c10a1027591bd5cc44a83e0ab50/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:26.480000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/33e560b130d782b733ad2050802881a70fb40c10a1027591bd5cc44a83e0ab50/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:26.901000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/33e560b130d782b733ad2050802881a70fb40c10a1027591bd5cc44a83e0ab50/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:27.008000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/33e560b130d782b733ad2050802881a70fb40c10a1027591bd5cc44a83e0ab50/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:27.113000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/33e560b130d782b733ad2050802881a70fb40c10a1027591bd5cc44a83e0ab50/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:27.114000 [ 588 ] DEBUG : ClickHouse node2 started (cluster.py:3159, start) 2025-11-13 09:09:27.115000 [ 588 ] DEBUG : get_instance_ip instance_name=node3 (cluster.py:2005, get_instance_ip) 2025-11-13 09:09:27.118000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestkeeperreconfigremovemany-gw6-node3-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:27.119000 [ 588 ] DEBUG : get_instance_ip instance_name=node3 (cluster.py:2015, get_instance_global_ipv6) 2025-11-13 09:09:27.121000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestkeeperreconfigremovemany-gw6-node3-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:27.122000 [ 588 ] DEBUG : Waiting for ClickHouse start in node3, ip: 172.16.5.5... (cluster.py:3155, start) 2025-11-13 09:09:27.124000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestkeeperreconfigremovemany-gw6-node3-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:27.127000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/6042baf3821d4f797c90a4289a0acc669a942164428cf2692fd84df8a3a35cf1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:27.128000 [ 588 ] DEBUG : ClickHouse node3 started (cluster.py:3159, start) 2025-11-13 09:09:27.129000 [ 588 ] DEBUG : get_instance_ip instance_name=node4 (cluster.py:2005, get_instance_ip) 2025-11-13 09:09:27.131000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestkeeperreconfigremovemany-gw6-node4-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:27.132000 [ 588 ] DEBUG : get_instance_ip instance_name=node4 (cluster.py:2015, get_instance_global_ipv6) 2025-11-13 09:09:27.134000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestkeeperreconfigremovemany-gw6-node4-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:27.135000 [ 588 ] DEBUG : Waiting for ClickHouse start in node4, ip: 172.16.5.2... (cluster.py:3155, start) 2025-11-13 09:09:27.137000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestkeeperreconfigremovemany-gw6-node4-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:27.139000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/0bf6a473c1da6827e121c48536814402c11e0cecd95754bb2f2a846939073706/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:27.244000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/0bf6a473c1da6827e121c48536814402c11e0cecd95754bb2f2a846939073706/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:27.354000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/0bf6a473c1da6827e121c48536814402c11e0cecd95754bb2f2a846939073706/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:27.461000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/0bf6a473c1da6827e121c48536814402c11e0cecd95754bb2f2a846939073706/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:27.567000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/0bf6a473c1da6827e121c48536814402c11e0cecd95754bb2f2a846939073706/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:27.673000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/0bf6a473c1da6827e121c48536814402c11e0cecd95754bb2f2a846939073706/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:27.780000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/0bf6a473c1da6827e121c48536814402c11e0cecd95754bb2f2a846939073706/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:27.891000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/0bf6a473c1da6827e121c48536814402c11e0cecd95754bb2f2a846939073706/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:27.896000 [ 588 ] DEBUG : ClickHouse node4 started (cluster.py:3159, start) 2025-11-13 09:09:27.896000 [ 588 ] DEBUG : get_instance_ip instance_name=node5 (cluster.py:2005, get_instance_ip) 2025-11-13 09:09:27.902000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestkeeperreconfigremovemany-gw6-node5-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:27.902000 [ 588 ] DEBUG : get_instance_ip instance_name=node5 (cluster.py:2015, get_instance_global_ipv6) 2025-11-13 09:09:27.936000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestkeeperreconfigremovemany-gw6-node5-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:27.937000 [ 588 ] DEBUG : Waiting for ClickHouse start in node5, ip: 172.16.5.3... (cluster.py:3155, start) 2025-11-13 09:09:27.944000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestkeeperreconfigremovemany-gw6-node5-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:27.949000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/619e05e071e840e8d406d13228d6838c49fa7e9c93c536fe5d88f64432972aa4/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:27.951000 [ 588 ] DEBUG : ClickHouse node5 started (cluster.py:3159, start) ----------------------------- Captured stdout call ----------------------------- Initial config server.1=node1:9234;participant;1 server.2=node2:9234;participant;1 server.3=node3:9234;participant;1 server.4=node4:9234;participant;1 server.5=node5:9234;participant;1 After removing 4,5 server.1=node1:9234;participant;1 server.2=node2:9234;participant;1 server.3=node3:9234;participant;1 ------------------------------ Captured log call ------------------------------- 2025-11-13 09:09:27.990000 [ 588 ] DEBUG : get_instance_ip instance_name=node1 (cluster.py:2005, get_instance_ip) 2025-11-13 09:09:28.003000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestkeeperreconfigremovemany-gw6-node1-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:31.928000 [ 588 ] DEBUG : get_instance_ip instance_name=node4 (cluster.py:2005, get_instance_ip) 2025-11-13 09:09:31.931000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestkeeperreconfigremovemany-gw6-node4-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:32.323000 [ 588 ] DEBUG : get_instance_ip instance_name=node5 (cluster.py:2005, get_instance_ip) 2025-11-13 09:09:32.328000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestkeeperreconfigremovemany-gw6-node5-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:40.149000 [ 588 ] DEBUG : get_instance_ip instance_name=node2 (cluster.py:2005, get_instance_ip) 2025-11-13 09:09:40.157000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestkeeperreconfigremovemany-gw6-node2-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:40.828000 [ 588 ] DEBUG : get_instance_ip instance_name=node1 (cluster.py:2005, get_instance_ip) 2025-11-13 09:09:40.837000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestkeeperreconfigremovemany-gw6-node1-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-11-13 09:09:42.637000 [ 588 ] DEBUG : run container_id:roottestkeeperreconfigremovemany-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', '[ -f /var/log/clickhouse-server/clickhouse-server.log ] && zgrep -aH "has been removed from the cluster" /var/log/clickhouse-server/clickhouse-server.log | ( [ -z "" ] && cat || grep -v "$" ) || true'] (cluster.py:2051, exec_in_container) 2025-11-13 09:09:42.638000 [ 588 ] DEBUG : Command:[docker exec roottestkeeperreconfigremovemany-gw6-node1-1 bash -c [ -f /var/log/clickhouse-server/clickhouse-server.log ] && zgrep -aH "has been removed from the cluster" /var/log/clickhouse-server/clickhouse-server.log | ( [ -z "" ] && cat || grep -v "$" ) || true] (cluster.py:121, run_and_check) 2025-11-13 09:09:43.129000 [ 588 ] DEBUG : run container_id:roottestkeeperreconfigremovemany-gw6-node2-1 detach:False nothrow:False cmd: ['bash', '-c', '[ -f /var/log/clickhouse-server/clickhouse-server.log ] && zgrep -aH "has been removed from the cluster" /var/log/clickhouse-server/clickhouse-server.log | ( [ -z "" ] && cat || grep -v "$" ) || true'] (cluster.py:2051, exec_in_container) 2025-11-13 09:09:43.130000 [ 588 ] DEBUG : Command:[docker exec roottestkeeperreconfigremovemany-gw6-node2-1 bash -c [ -f /var/log/clickhouse-server/clickhouse-server.log ] && zgrep -aH "has been removed from the cluster" /var/log/clickhouse-server/clickhouse-server.log | ( [ -z "" ] && cat || grep -v "$" ) || true] (cluster.py:121, run_and_check) 2025-11-13 09:09:43.916000 [ 588 ] DEBUG : run container_id:roottestkeeperreconfigremovemany-gw6-node3-1 detach:False nothrow:False cmd: ['bash', '-c', '[ -f /var/log/clickhouse-server/clickhouse-server.log ] && zgrep -aH "has been removed from the cluster" /var/log/clickhouse-server/clickhouse-server.log | ( [ -z "" ] && cat || grep -v "$" ) || true'] (cluster.py:2051, exec_in_container) 2025-11-13 09:09:43.917000 [ 588 ] DEBUG : Command:[docker exec roottestkeeperreconfigremovemany-gw6-node3-1 bash -c [ -f /var/log/clickhouse-server/clickhouse-server.log ] && zgrep -aH "has been removed from the cluster" /var/log/clickhouse-server/clickhouse-server.log | ( [ -z "" ] && cat || grep -v "$" ) || true] (cluster.py:121, run_and_check) 2025-11-13 09:09:44.393000 [ 588 ] DEBUG : run container_id:roottestkeeperreconfigremovemany-gw6-node4-1 detach:False nothrow:False cmd: ['bash', '-c', '[ -f /var/log/clickhouse-server/clickhouse-server.log ] && zgrep -aH "has been removed from the cluster" /var/log/clickhouse-server/clickhouse-server.log | ( [ -z "" ] && cat || grep -v "$" ) || true'] (cluster.py:2051, exec_in_container) 2025-11-13 09:09:44.393000 [ 588 ] DEBUG : Command:[docker exec roottestkeeperreconfigremovemany-gw6-node4-1 bash -c [ -f /var/log/clickhouse-server/clickhouse-server.log ] && zgrep -aH "has been removed from the cluster" /var/log/clickhouse-server/clickhouse-server.log | ( [ -z "" ] && cat || grep -v "$" ) || true] (cluster.py:121, run_and_check) 2025-11-13 09:09:44.676000 [ 588 ] DEBUG : Stdout:/var/log/clickhouse-server/clickhouse-server.log:2025.11.13 09:09:40.468763 [ 602 ] {} RaftInstance: this server (4) has been removed from the cluster, will step down itself soon. config log idx 272 (cluster.py:145, run_and_check) 2025-11-13 09:09:44.676000 [ 588 ] DEBUG : run container_id:roottestkeeperreconfigremovemany-gw6-node5-1 detach:False nothrow:False cmd: ['bash', '-c', '[ -f /var/log/clickhouse-server/clickhouse-server.log ] && zgrep -aH "has been removed from the cluster" /var/log/clickhouse-server/clickhouse-server.log | ( [ -z "" ] && cat || grep -v "$" ) || true'] (cluster.py:2051, exec_in_container) 2025-11-13 09:09:44.677000 [ 588 ] DEBUG : Command:[docker exec roottestkeeperreconfigremovemany-gw6-node5-1 bash -c [ -f /var/log/clickhouse-server/clickhouse-server.log ] && zgrep -aH "has been removed from the cluster" /var/log/clickhouse-server/clickhouse-server.log | ( [ -z "" ] && cat || grep -v "$" ) || true] (cluster.py:121, run_and_check) 2025-11-13 09:09:44.941000 [ 588 ] DEBUG : Stdout:/var/log/clickhouse-server/clickhouse-server.log:2025.11.13 09:09:40.579224 [ 600 ] {} RaftInstance: this server (5) has been removed from the cluster, will step down itself soon. config log idx 274 (cluster.py:145, run_and_check) 2025-11-13 09:09:44.942000 [ 588 ] DEBUG : Sending stat to :9181 (keeper_utils.py:271, send_4lw_cmd) 2025-11-13 09:09:44.942000 [ 588 ] DEBUG : get_instance_ip instance_name=node1 (cluster.py:2005, get_instance_ip) 2025-11-13 09:09:44.944000 [ 588 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestkeeperreconfigremovemany-gw6-node1-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) ---------------------------- Captured log teardown ----------------------------- 2025-11-13 09:09:52.783000 [ 588 ] DEBUG : Command:[docker compose --env-file /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/.env --project-name roottestkeeperreconfigremovemany-gw6 --file /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node1/docker-compose.yml --file /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node2/docker-compose.yml --file /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node3/docker-compose.yml --file /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node4/docker-compose.yml --file /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node5/docker-compose.yml stop --timeout 20] (cluster.py:121, run_and_check) 2025-11-13 09:10:11.145000 [ 588 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node4-1 Stopping (cluster.py:147, run_and_check) 2025-11-13 09:10:11.145000 [ 588 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node5-1 Stopping (cluster.py:147, run_and_check) 2025-11-13 09:10:11.145000 [ 588 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node1-1 Stopping (cluster.py:147, run_and_check) 2025-11-13 09:10:11.145000 [ 588 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node2-1 Stopping (cluster.py:147, run_and_check) 2025-11-13 09:10:11.145000 [ 588 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node3-1 Stopping (cluster.py:147, run_and_check) 2025-11-13 09:10:11.145000 [ 588 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node2-1 Stopped (cluster.py:147, run_and_check) 2025-11-13 09:10:11.145000 [ 588 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node3-1 Stopped (cluster.py:147, run_and_check) 2025-11-13 09:10:11.146000 [ 588 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node4-1 Stopped (cluster.py:147, run_and_check) 2025-11-13 09:10:11.146000 [ 588 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node5-1 Stopped (cluster.py:147, run_and_check) 2025-11-13 09:10:11.146000 [ 588 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node1-1 Stopped (cluster.py:147, run_and_check) 2025-11-13 09:10:11.146000 [ 588 ] DEBUG : Command:[bash -c [ -f /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node1/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node1/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true] (cluster.py:121, run_and_check) 2025-11-13 09:10:11.159000 [ 588 ] DEBUG : Command:[bash -c [ -f /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node2/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node2/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true] (cluster.py:121, run_and_check) 2025-11-13 09:10:11.176000 [ 588 ] DEBUG : Command:[bash -c [ -f /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node3/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node3/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true] (cluster.py:121, run_and_check) 2025-11-13 09:10:11.189000 [ 588 ] DEBUG : Command:[bash -c [ -f /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node4/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node4/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true] (cluster.py:121, run_and_check) 2025-11-13 09:10:11.202000 [ 588 ] DEBUG : Command:[bash -c [ -f /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node5/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node5/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true] (cluster.py:121, run_and_check) 2025-11-13 09:10:11.214000 [ 588 ] DEBUG : Command:[docker compose --env-file /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/.env --project-name roottestkeeperreconfigremovemany-gw6 --file /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node1/docker-compose.yml --file /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node2/docker-compose.yml --file /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node3/docker-compose.yml --file /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node4/docker-compose.yml --file /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node5/docker-compose.yml down --volumes] (cluster.py:121, run_and_check) 2025-11-13 09:10:11.948000 [ 588 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node1-1 Stopping (cluster.py:147, run_and_check) 2025-11-13 09:10:11.948000 [ 588 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node3-1 Stopping (cluster.py:147, run_and_check) 2025-11-13 09:10:11.948000 [ 588 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node4-1 Stopping (cluster.py:147, run_and_check) 2025-11-13 09:10:11.948000 [ 588 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node2-1 Stopping (cluster.py:147, run_and_check) 2025-11-13 09:10:11.948000 [ 588 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node5-1 Stopping (cluster.py:147, run_and_check) 2025-11-13 09:10:11.948000 [ 588 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node1-1 Stopped (cluster.py:147, run_and_check) 2025-11-13 09:10:11.948000 [ 588 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node1-1 Removing (cluster.py:147, run_and_check) 2025-11-13 09:10:11.949000 [ 588 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node4-1 Stopped (cluster.py:147, run_and_check) 2025-11-13 09:10:11.949000 [ 588 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node4-1 Removing (cluster.py:147, run_and_check) 2025-11-13 09:10:11.949000 [ 588 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node2-1 Stopped (cluster.py:147, run_and_check) 2025-11-13 09:10:11.949000 [ 588 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node2-1 Removing (cluster.py:147, run_and_check) 2025-11-13 09:10:11.949000 [ 588 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node5-1 Stopped (cluster.py:147, run_and_check) 2025-11-13 09:10:11.949000 [ 588 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node5-1 Removing (cluster.py:147, run_and_check) 2025-11-13 09:10:11.949000 [ 588 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node3-1 Stopped (cluster.py:147, run_and_check) 2025-11-13 09:10:11.949000 [ 588 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node3-1 Removing (cluster.py:147, run_and_check) 2025-11-13 09:10:11.949000 [ 588 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node2-1 Removed (cluster.py:147, run_and_check) 2025-11-13 09:10:11.949000 [ 588 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node1-1 Removed (cluster.py:147, run_and_check) 2025-11-13 09:10:11.949000 [ 588 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node4-1 Removed (cluster.py:147, run_and_check) 2025-11-13 09:10:11.949000 [ 588 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node3-1 Removed (cluster.py:147, run_and_check) 2025-11-13 09:10:11.949000 [ 588 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node5-1 Removed (cluster.py:147, run_and_check) 2025-11-13 09:10:11.949000 [ 588 ] DEBUG : Stderr: Network roottestkeeperreconfigremovemany-gw6_default Removing (cluster.py:147, run_and_check) 2025-11-13 09:10:11.949000 [ 588 ] DEBUG : Stderr: Network roottestkeeperreconfigremovemany-gw6_default Removed (cluster.py:147, run_and_check) 2025-11-13 09:10:11.950000 [ 588 ] DEBUG : Cleanup called (cluster.py:851, cleanup) 2025-11-13 09:10:11.971000 [ 588 ] DEBUG : Docker networks for project roottestkeeperreconfigremovemany-gw6 are NETWORK ID NAME DRIVER SCOPE (cluster.py:830, print_all_docker_pieces) 2025-11-13 09:10:11.996000 [ 588 ] DEBUG : Docker containers for project roottestkeeperreconfigremovemany-gw6 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES (cluster.py:838, print_all_docker_pieces) 2025-11-13 09:10:12.024000 [ 588 ] DEBUG : Docker volumes for project roottestkeeperreconfigremovemany-gw6 are DRIVER VOLUME NAME (cluster.py:846, print_all_docker_pieces) 2025-11-13 09:10:12.024000 [ 588 ] DEBUG : Command:[docker container list --all --filter name='^/roottestkeeperreconfigremovemany-gw6-.*-1$' --format '{{.ID}}:{{.Names}}'] (cluster.py:121, run_and_check) 2025-11-13 09:10:12.049000 [ 588 ] DEBUG : Unstopped containers: {} (cluster.py:865, cleanup) 2025-11-13 09:10:12.049000 [ 588 ] DEBUG : No running containers for project: roottestkeeperreconfigremovemany-gw6 (cluster.py:879, cleanup) 2025-11-13 09:10:12.050000 [ 588 ] DEBUG : Trying to prune unused networks... (cluster.py:885, cleanup) 2025-11-13 09:10:12.071000 [ 588 ] DEBUG : Trying to prune unused images... (cluster.py:901, cleanup) 2025-11-13 09:10:12.071000 [ 588 ] DEBUG : Command:[docker image prune -f] (cluster.py:121, run_and_check) 2025-11-13 09:10:12.120000 [ 588 ] DEBUG : Stdout:Total reclaimed space: 0B (cluster.py:145, run_and_check) 2025-11-13 09:10:12.120000 [ 588 ] DEBUG : Images pruned (cluster.py:904, cleanup) 2025-11-13 09:10:12.121000 [ 588 ] DEBUG : Trying to prune unused volumes... (cluster.py:910, cleanup) 2025-11-13 09:10:12.121000 [ 588 ] DEBUG : Command:[docker volume ls | wc -l] (cluster.py:121, run_and_check) 2025-11-13 09:10:12.155000 [ 588 ] DEBUG : Stdout:5 (cluster.py:145, run_and_check) 2025-11-13 09:10:12.155000 [ 588 ] DEBUG : Command:[docker volume prune -f] (cluster.py:121, run_and_check) 2025-11-13 09:10:12.180000 [ 588 ] DEBUG : Stdout:Total reclaimed space: 0B (cluster.py:145, run_and_check) 2025-11-13 09:10:12.181000 [ 588 ] DEBUG : Volumes pruned: 5 (cluster.py:915, cleanup) ----------------- generated report log file: parallel1_0.jsonl ----------------- ============================== slowest durations =============================== 392.78s call test_log_query_probability/test.py::test_log_quries_probability_two 177.48s call test_keeper_snapshot_small_distance/test.py::test_snapshot_and_load 90.43s call test_log_query_probability/test.py::test_log_quries_probability_one 82.49s call test_polymorphic_parts/test.py::test_compact_parts_only 74.00s call test_polymorphic_parts/test.py::test_polymorphic_parts_basics[first_node0-second_node0] 60.24s call test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-101-10000-SELECT sum(key) FROM {table_name}] 52.63s teardown test_format_avro_confluent/test.py::test_select_auth_encoded_complex 52.00s call test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-11-1000-SELECT sum(key) FROM {table_name}] 49.39s setup test_keeper_profiler/test.py::test_profiler 46.04s call test_profile_events_s3/test.py::test_profile_events 43.64s call test_keeper_restore_from_snapshot/test_disk_s3.py::test_recover_from_snapshot_with_disk_s3 41.29s setup test_format_avro_confluent/test.py::test_select 40.80s call test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-101-100000-SELECT sum(key) FROM {table_name}] 40.72s setup test_groupBitmapAnd_on_distributed/test_groupBitmapAndState_on_distributed_table.py::test_groupBitmapAndState_on_distributed_table 40.18s call test_polymorphic_parts/test.py::test_polymorphic_parts_basics[first_node1-second_node1] 37.26s call test_executable_dictionary/test.py::test_executable_source_updated_python 35.14s call test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_mergetree[test_single_shard_multiple_replicas-custom_key_sampling-key] 32.82s setup test_keeper_restore_from_snapshot/test_disk_s3.py::test_recover_from_snapshot_with_disk_s3 31.27s setup test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-None-False-False] 30.07s setup test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-None-True-False] 29.91s teardown test_kerberos_auth/test.py::test_kerberos_auth_without_keytab 28.55s setup test_keeper_reconfig_remove_many/test.py::test_reconfig_remove_2_and_leader 27.62s setup test_profile_events_s3/test.py::test_profile_events 27.07s setup test_polymorphic_parts/test.py::test_compact_parts_only 26.67s setup test_parallel_replicas_increase_error_count/test.py::test_increase_error_count 25.15s call test_groupBitmapAnd_on_distributed/test_groupBitmapAndState_on_distributed_table.py::test_groupBitmapAndState_on_distributed_table 24.01s teardown test_keeper_profiler/test.py::test_profiler 23.99s teardown test_keeper_restore_from_snapshot/test_disk_s3.py::test_recover_from_snapshot_with_disk_s3 23.27s call test_keeper_reconfig_remove_many/test.py::test_reconfig_remove_2_and_leader 22.65s setup test_executable_dictionary/test.py::test_executable_input_bash 22.35s teardown test_profile_events_s3/test.py::test_profile_events 21.81s teardown test_merge_tree_check_part_with_cache/test.py::test_check_part_with_cache 20.63s teardown test_keeper_reconfig_remove_many/test.py::test_reconfig_remove_2_and_leader 19.37s setup test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_distributed[test_multiple_shards_multiple_replicas-custom_key_sampling-key] 18.41s setup test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-101-1000-SELECT sum(key) FROM {table_name}] 18.32s setup test_keeper_snapshot_small_distance/test.py::test_snapshot_and_load 17.22s call test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-11-100000-SELECT sum(key) FROM {table_name}] 16.61s call test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_replicatedmergetree[test_single_shard_multiple_replicas-custom_key_sampling-sipHash64(key)] 16.47s teardown test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_replicatedmergetree[test_single_shard_multiple_replicas-custom_key_sampling-sipHash64(key)] 16.18s setup test_kerberos_auth/test.py::test_bad_path_to_keytab 15.77s setup test_match_process_uid_against_data_owner/test.py::test_different_user 15.73s call test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_mergetree[test_single_shard_multiple_replicas-custom_key_sampling-sipHash64(key)] 15.68s teardown test_groupBitmapAnd_on_distributed/test_groupBitmapAndState_on_distributed_table.py::test_groupBitmapAndState_on_distributed_table 15.20s call test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-101-1000-SELECT sum(key) FROM {table_name}] 14.80s setup test_merge_tree_check_part_with_cache/test.py::test_check_part_with_cache 14.79s setup test_max_authentication_methods_per_user/test.py::test_alter 14.69s setup test_password_constraints/test.py::test_complexity_rules 14.38s call test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-depends_on1-False-False] 14.28s call test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-101-1000-SELECT sum(key) FROM {table_name}] 13.81s call test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-None-False-True] 13.42s setup test_log_query_probability/test.py::test_log_quries_probability_one 13.40s teardown test_polymorphic_parts/test.py::test_polymorphic_parts_non_adaptive 13.16s setup test_log_levels_update/test.py::test_log_levels_update 12.93s call test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-None-True-False] 12.83s setup test_grant_and_revoke/test_without_table_engine_grant.py::test_table_engine_and_source_grant 12.29s call test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-101-100000-SELECT sum(key) FROM {table_name}] 11.85s call test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_mergetree[test_single_shard_multiple_replicas-custom_key_range-sipHash64(key)] 11.85s teardown test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-11-100000-SELECT sum(key) FROM {table_name}] 11.52s call test_executable_dictionary/test.py::test_executable_input_slow_python 11.46s call test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-11-10000-SELECT sum(key) FROM {table_name}] 11.26s call test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-101-10000-SELECT sum(key) FROM {table_name}] 10.99s call test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-11-10000-SELECT sum(key) FROM {table_name}] 10.98s call test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_replicatedmergetree[test_single_shard_multiple_replicas-custom_key_range-key] 10.92s call test_executable_dictionary/test.py::test_executable_source_python 10.56s call test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_replicatedmergetree[test_single_shard_multiple_replicas-custom_key_sampling-key] 10.51s call test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-11-100000-SELECT sum(key) FROM {table_name}] 10.41s call test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-11-1000-SELECT sum(key) FROM {table_name}] 10.33s call test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-depends_on1-False-True] 10.19s call test_match_process_uid_against_data_owner/test.py::test_different_user 9.88s setup test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-depends_on1-True-False] 9.54s setup test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-depends_on1-False-True] 9.06s call test_executable_dictionary/test.py::test_executable_source_argument_python 8.95s call test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_replicatedmergetree[test_single_shard_multiple_replicas-custom_key_range-sipHash64(key)] 7.93s call test_polymorphic_parts/test.py::test_different_part_types_on_replicas[polymorphic_table_compact-Compact] 7.81s teardown test_parallel_replicas_increase_error_count/test.py::test_increase_error_count 7.60s call test_parallel_replicas_increase_error_count/test.py::test_increase_error_count 7.46s call test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-None-True-True] 7.17s teardown test_log_query_probability/test.py::test_log_quries_probability_two 7.02s setup test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-None-False-True] 6.99s call test_executable_dictionary/test.py::test_executable_source_exit_code_check 6.25s setup test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-None-True-True] 6.02s call test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-None-False-False] 5.96s call test_polymorphic_parts/test.py::test_different_part_types_on_replicas[polymorphic_table_wide-Wide] 5.94s teardown test_max_authentication_methods_per_user/test.py::test_create_default_setting 5.85s call test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_distributed[test_single_shard_multiple_replicas-custom_key_sampling-sipHash64(key)] 5.71s call test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-None-True-False] 5.64s call test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-depends_on1-True-False] 5.56s call test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_mergetree[test_single_shard_multiple_replicas-custom_key_range-key] 5.55s call test_format_avro_confluent/test.py::test_select_auth 5.52s teardown test_password_constraints/test.py::test_default_password_type 5.29s teardown test_executable_dictionary/test.py::test_executable_source_updated_python 4.89s call test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-None-True-True] 4.51s setup test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-depends_on1-False-False] 4.49s teardown test_keeper_snapshot_small_distance/test.py::test_snapshot_and_load 4.20s call test_log_levels_update/test.py::test_log_levels_update 4.18s call test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-depends_on1-True-True] 4.01s setup test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-depends_on1-False-False] 3.89s teardown test_grant_and_revoke/test_without_table_engine_grant.py::test_table_engine_and_source_grant 3.65s call test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_distributed[test_multiple_shards_multiple_replicas-custom_key_sampling-key] 3.61s call test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-depends_on1-False-False] 3.55s teardown test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-depends_on1-True-True] 3.51s call test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-depends_on1-True-True] 3.50s call test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-None-False-True] 3.50s call test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-depends_on1-True-False] 3.48s setup test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-None-True-True] 3.40s call test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-None-False-False] 3.40s call test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-None-False-False] 3.35s call test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-depends_on1-False-True] 3.34s call test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-depends_on1-False-True] 3.34s call test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-depends_on1-False-False] 3.34s call test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-depends_on1-True-True] 3.33s teardown test_log_levels_update/test.py::test_log_levels_update 3.24s call test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-depends_on1-True-False] 3.20s setup test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-depends_on1-True-True] 3.19s call test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-None-False-True] 3.14s call test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-depends_on1-True-True] 3.11s call test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_distributed[test_multiple_shards_multiple_replicas-custom_key_sampling-sipHash64(key)] 3.10s call test_format_avro_confluent/test.py::test_select 3.09s call test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-depends_on1-False-True] 3.09s call test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-None-False-False] 3.09s call test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-None-True-False] 3.09s call test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-depends_on1-True-False] 3.04s call test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-None-True-True] 3.04s call test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-depends_on1-False-False] 3.04s call test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-None-True-False] 3.04s call test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-None-False-True] 2.99s call test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-None-True-True] 2.94s setup test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-None-True-False] 2.94s call test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_distributed[test_single_shard_multiple_replicas-custom_key_sampling-key] 2.85s teardown test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-None-False-True] 2.69s call test_grant_and_revoke/test_without_table_engine_grant.py::test_table_engine_and_source_grant 2.67s setup test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-depends_on1-True-True] 2.65s call test_polymorphic_parts/test.py::test_polymorphic_parts_non_adaptive 2.60s call test_format_avro_confluent/test.py::test_select_auth_encoded_complex 2.56s call test_password_constraints/test.py::test_complexity_rules 2.56s setup test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-None-True-False] 2.42s teardown test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-None-True-False] 2.41s setup test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-None-False-False] 2.38s setup test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-depends_on1-False-True] 2.37s setup test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-None-False-True] 2.36s setup test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-None-True-True] 2.32s setup test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-depends_on1-True-False] 2.27s setup test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-None-False-False] 2.21s setup test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-depends_on1-False-False] 2.21s setup test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-depends_on1-True-True] 2.21s setup test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-None-False-True] 2.17s setup test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-depends_on1-True-False] 2.11s setup test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-depends_on1-True-False] 2.07s call test_polymorphic_parts/test.py::test_polymorphic_parts_index 2.06s setup test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-depends_on1-False-True] 2.06s setup test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-depends_on1-True-True] 2.06s setup test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-None-True-False] 2.02s teardown test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-depends_on1-False-False] 2.01s setup test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-None-False-False] 2.01s setup test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-depends_on1-False-True] 2.01s setup test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-None-False-True] 1.96s setup test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-depends_on1-False-False] 1.96s setup test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-None-True-True] 1.89s call test_max_authentication_methods_per_user/test.py::test_alter 1.57s call test_max_authentication_methods_per_user/test.py::test_alter_default_setting 1.52s call test_executable_dictionary/test.py::test_executable_non_direct_input_bash 1.49s call test_executable_dictionary/test.py::test_executable_input_sum_python 1.43s teardown test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-None-True-True] 1.38s teardown test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-depends_on1-False-True] 1.31s teardown test_match_process_uid_against_data_owner/test.py::test_different_user 1.29s call test_max_authentication_methods_per_user/test.py::test_create 1.23s teardown test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-None-True-False] 1.19s call test_max_authentication_methods_per_user/test.py::test_create_default_setting 1.10s call test_format_avro_confluent/test.py::test_select_auth_encoded 1.00s teardown test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-None-False-False] 0.98s call test_executable_dictionary/test.py::test_executable_input_signalled_python 0.86s call test_executable_dictionary/test.py::test_executable_input_bash 0.86s teardown test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-None-True-True] 0.85s call test_executable_dictionary/test.py::test_executable_input_python 0.80s call test_executable_dictionary/test.py::test_executable_input_send_chunk_header_python 0.80s call test_keeper_profiler/test.py::test_profiler 0.79s teardown test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-None-True-False] 0.73s teardown test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-None-False-False] 0.72s teardown test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-depends_on1-True-False] 0.70s call test_kerberos_auth/test.py::test_bad_path_to_keytab 0.70s teardown test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-None-False-True] 0.68s call test_password_constraints/test.py::test_default_password_type 0.63s teardown test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-depends_on1-False-True] 0.59s teardown test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-depends_on1-False-False] 0.58s teardown test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-depends_on1-True-True] 0.58s teardown test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-depends_on1-True-False] 0.58s teardown test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-depends_on1-True-True] 0.58s teardown test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-depends_on1-False-False] 0.58s teardown test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-depends_on1-True-False] 0.58s teardown test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-None-True-True] 0.54s teardown test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-None-False-False] 0.53s teardown test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-None-True-False] 0.53s teardown test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-depends_on1-True-False] 0.53s teardown test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-None-True-True] 0.53s teardown test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-None-False-True] 0.53s teardown test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-None-False-False] 0.53s teardown test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-depends_on1-True-True] 0.53s teardown test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-depends_on1-False-False] 0.53s teardown test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-depends_on1-False-True] 0.49s teardown test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-depends_on1-False-True] 0.48s teardown test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-None-False-True] 0.37s call test_kerberos_auth/test.py::test_kerberos_auth_without_keytab 0.27s call test_merge_tree_check_part_with_cache/test.py::test_check_part_with_cache 0.25s call test_kerberos_auth/test.py::test_kerberos_auth_with_keytab 0.05s setup test_executable_dictionary/test.py::test_executable_input_sum_python 0.02s teardown test_polymorphic_parts/test.py::test_compact_parts_only 0.01s teardown test_format_avro_confluent/test.py::test_select_auth 0.01s teardown test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-101-100000-SELECT sum(key) FROM {table_name}] 0.01s teardown test_executable_dictionary/test.py::test_executable_source_argument_python 0.01s teardown test_format_avro_confluent/test.py::test_select 0.01s teardown test_polymorphic_parts/test.py::test_different_part_types_on_replicas[polymorphic_table_compact-Compact] 0.01s teardown test_log_query_probability/test.py::test_log_quries_probability_one 0.01s teardown test_format_avro_confluent/test.py::test_select_auth_encoded 0.01s teardown test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_replicatedmergetree[test_single_shard_multiple_replicas-custom_key_sampling-key] 0.01s teardown test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-101-10000-SELECT sum(key) FROM {table_name}] 0.01s setup test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_mergetree[test_single_shard_multiple_replicas-custom_key_sampling-sipHash64(key)] 0.01s setup test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-101-100000-SELECT sum(key) FROM {table_name}] 0.01s setup test_format_avro_confluent/test.py::test_select_auth_encoded 0.01s teardown test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_mergetree[test_single_shard_multiple_replicas-custom_key_sampling-key] 0.01s setup test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_replicatedmergetree[test_single_shard_multiple_replicas-custom_key_sampling-key] 0.00s teardown test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_mergetree[test_single_shard_multiple_replicas-custom_key_range-sipHash64(key)] 0.00s teardown test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-101-1000-SELECT sum(key) FROM {table_name}] 0.00s setup test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_distributed[test_single_shard_multiple_replicas-custom_key_sampling-key] 0.00s teardown test_polymorphic_parts/test.py::test_polymorphic_parts_basics[first_node1-second_node1] 0.00s setup test_polymorphic_parts/test.py::test_different_part_types_on_replicas[polymorphic_table_compact-Compact] 0.00s teardown test_polymorphic_parts/test.py::test_polymorphic_parts_basics[first_node0-second_node0] 0.00s setup test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_mergetree[test_single_shard_multiple_replicas-custom_key_sampling-key] 0.00s teardown test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-11-100000-SELECT sum(key) FROM {table_name}] 0.00s setup test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-11-10000-SELECT sum(key) FROM {table_name}] 0.00s teardown test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-11-1000-SELECT sum(key) FROM {table_name}] 0.00s setup test_polymorphic_parts/test.py::test_polymorphic_parts_basics[first_node1-second_node1] 0.00s teardown test_executable_dictionary/test.py::test_executable_input_slow_python 0.00s setup test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-101-10000-SELECT sum(key) FROM {table_name}] 0.00s setup test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-11-1000-SELECT sum(key) FROM {table_name}] 0.00s setup test_log_query_probability/test.py::test_log_quries_probability_two 0.00s setup test_format_avro_confluent/test.py::test_select_auth 0.00s teardown test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_mergetree[test_single_shard_multiple_replicas-custom_key_range-key] 0.00s setup test_executable_dictionary/test.py::test_executable_source_exit_code_check 0.00s setup test_polymorphic_parts/test.py::test_polymorphic_parts_index 0.00s setup test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_replicatedmergetree[test_single_shard_multiple_replicas-custom_key_sampling-sipHash64(key)] 0.00s setup test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-11-100000-SELECT sum(key) FROM {table_name}] 0.00s teardown test_polymorphic_parts/test.py::test_different_part_types_on_replicas[polymorphic_table_wide-Wide] 0.00s setup test_executable_dictionary/test.py::test_executable_source_python 0.00s setup test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-101-1000-SELECT sum(key) FROM {table_name}] 0.00s setup test_executable_dictionary/test.py::test_executable_input_slow_python 0.00s setup test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_replicatedmergetree[test_single_shard_multiple_replicas-custom_key_range-key] 0.00s teardown test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-11-10000-SELECT sum(key) FROM {table_name}] 0.00s setup test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_distributed[test_single_shard_multiple_replicas-custom_key_sampling-sipHash64(key)] 0.00s setup test_format_avro_confluent/test.py::test_select_auth_encoded_complex 0.00s teardown test_executable_dictionary/test.py::test_executable_source_exit_code_check 0.00s teardown test_max_authentication_methods_per_user/test.py::test_create 0.00s setup test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-11-10000-SELECT sum(key) FROM {table_name}] 0.00s setup test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-101-10000-SELECT sum(key) FROM {table_name}] 0.00s setup test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_mergetree[test_single_shard_multiple_replicas-custom_key_range-sipHash64(key)] 0.00s setup test_executable_dictionary/test.py::test_executable_source_updated_python 0.00s teardown test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_replicatedmergetree[test_single_shard_multiple_replicas-custom_key_range-sipHash64(key)] 0.00s setup test_polymorphic_parts/test.py::test_different_part_types_on_replicas[polymorphic_table_wide-Wide] 0.00s setup test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_replicatedmergetree[test_single_shard_multiple_replicas-custom_key_range-sipHash64(key)] 0.00s teardown test_executable_dictionary/test.py::test_executable_source_python 0.00s setup test_polymorphic_parts/test.py::test_polymorphic_parts_basics[first_node0-second_node0] 0.00s setup test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_distributed[test_multiple_shards_multiple_replicas-custom_key_sampling-sipHash64(key)] 0.00s setup test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-11-100000-SELECT sum(key) FROM {table_name}] 0.00s setup test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-11-1000-SELECT sum(key) FROM {table_name}] 0.00s teardown test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-101-10000-SELECT sum(key) FROM {table_name}] 0.00s teardown test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-101-1000-SELECT sum(key) FROM {table_name}] 0.00s setup test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-101-100000-SELECT sum(key) FROM {table_name}] 0.00s teardown test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-11-10000-SELECT sum(key) FROM {table_name}] 0.00s teardown test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_distributed[test_multiple_shards_multiple_replicas-custom_key_sampling-key] 0.00s teardown test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_replicatedmergetree[test_single_shard_multiple_replicas-custom_key_range-key] 0.00s teardown test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_mergetree[test_single_shard_multiple_replicas-custom_key_sampling-sipHash64(key)] 0.00s setup test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_mergetree[test_single_shard_multiple_replicas-custom_key_range-key] 0.00s setup test_polymorphic_parts/test.py::test_polymorphic_parts_non_adaptive 0.00s setup test_executable_dictionary/test.py::test_executable_source_argument_python 0.00s setup test_kerberos_auth/test.py::test_kerberos_auth_with_keytab 0.00s setup test_max_authentication_methods_per_user/test.py::test_alter_default_setting 0.00s teardown test_max_authentication_methods_per_user/test.py::test_alter 0.00s setup test_executable_dictionary/test.py::test_executable_non_direct_input_bash 0.00s teardown test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-11-1000-SELECT sum(key) FROM {table_name}] 0.00s teardown test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-101-100000-SELECT sum(key) FROM {table_name}] 0.00s setup test_max_authentication_methods_per_user/test.py::test_create_default_setting 0.00s teardown test_executable_dictionary/test.py::test_executable_non_direct_input_bash 0.00s setup test_executable_dictionary/test.py::test_executable_input_python 0.00s teardown test_executable_dictionary/test.py::test_executable_input_bash 0.00s teardown test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_distributed[test_single_shard_multiple_replicas-custom_key_sampling-sipHash64(key)] 0.00s setup test_executable_dictionary/test.py::test_executable_input_send_chunk_header_python 0.00s teardown test_polymorphic_parts/test.py::test_polymorphic_parts_index 0.00s teardown test_kerberos_auth/test.py::test_bad_path_to_keytab 0.00s teardown test_password_constraints/test.py::test_complexity_rules 0.00s setup test_max_authentication_methods_per_user/test.py::test_create 0.00s teardown test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_distributed[test_multiple_shards_multiple_replicas-custom_key_sampling-sipHash64(key)] 0.00s teardown test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_distributed[test_single_shard_multiple_replicas-custom_key_sampling-key] 0.00s teardown test_executable_dictionary/test.py::test_executable_input_signalled_python 0.00s teardown test_executable_dictionary/test.py::test_executable_input_sum_python 0.00s setup test_executable_dictionary/test.py::test_executable_input_signalled_python 0.00s teardown test_max_authentication_methods_per_user/test.py::test_alter_default_setting 0.00s setup test_kerberos_auth/test.py::test_kerberos_auth_without_keytab 0.00s setup test_password_constraints/test.py::test_default_password_type 0.00s teardown test_executable_dictionary/test.py::test_executable_input_send_chunk_header_python 0.00s teardown test_executable_dictionary/test.py::test_executable_input_python 0.00s teardown test_kerberos_auth/test.py::test_kerberos_auth_with_keytab =========================== short test summary info ============================ FAILED test_keeper_reconfig_remove_many/test.py::test_reconfig_remove_2_and_leader PASSED test_max_authentication_methods_per_user/test.py::test_alter PASSED test_kerberos_auth/test.py::test_bad_path_to_keytab PASSED test_kerberos_auth/test.py::test_kerberos_auth_with_keytab PASSED test_password_constraints/test.py::test_complexity_rules PASSED test_kerberos_auth/test.py::test_kerberos_auth_without_keytab PASSED test_password_constraints/test.py::test_default_password_type PASSED test_max_authentication_methods_per_user/test.py::test_alter_default_setting PASSED test_max_authentication_methods_per_user/test.py::test_create PASSED test_max_authentication_methods_per_user/test.py::test_create_default_setting PASSED test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_distributed[test_multiple_shards_multiple_replicas-custom_key_sampling-key] PASSED test_executable_dictionary/test.py::test_executable_input_bash PASSED test_executable_dictionary/test.py::test_executable_input_python PASSED test_executable_dictionary/test.py::test_executable_input_send_chunk_header_python PASSED test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_distributed[test_multiple_shards_multiple_replicas-custom_key_sampling-sipHash64(key)] PASSED test_executable_dictionary/test.py::test_executable_input_signalled_python PASSED test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_distributed[test_single_shard_multiple_replicas-custom_key_sampling-key] PASSED test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-101-1000-SELECT sum(key) FROM {table_name}] PASSED test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_distributed[test_single_shard_multiple_replicas-custom_key_sampling-sipHash64(key)] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-None-False-False] PASSED test_executable_dictionary/test.py::test_executable_input_slow_python PASSED test_executable_dictionary/test.py::test_executable_input_sum_python PASSED test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_mergetree[test_single_shard_multiple_replicas-custom_key_range-key] PASSED test_executable_dictionary/test.py::test_executable_non_direct_input_bash PASSED test_format_avro_confluent/test.py::test_select PASSED test_executable_dictionary/test.py::test_executable_source_argument_python PASSED test_format_avro_confluent/test.py::test_select_auth PASSED test_format_avro_confluent/test.py::test_select_auth_encoded PASSED test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_mergetree[test_single_shard_multiple_replicas-custom_key_range-sipHash64(key)] PASSED test_format_avro_confluent/test.py::test_select_auth_encoded_complex PASSED test_executable_dictionary/test.py::test_executable_source_exit_code_check PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-None-False-True] PASSED test_executable_dictionary/test.py::test_executable_source_python PASSED test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_mergetree[test_single_shard_multiple_replicas-custom_key_sampling-key] PASSED test_groupBitmapAnd_on_distributed/test_groupBitmapAndState_on_distributed_table.py::test_groupBitmapAndState_on_distributed_table PASSED test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-101-10000-SELECT sum(key) FROM {table_name}] PASSED test_keeper_profiler/test.py::test_profiler PASSED test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_mergetree[test_single_shard_multiple_replicas-custom_key_sampling-sipHash64(key)] PASSED test_log_query_probability/test.py::test_log_quries_probability_one PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-None-True-False] PASSED test_executable_dictionary/test.py::test_executable_source_updated_python PASSED test_polymorphic_parts/test.py::test_compact_parts_only PASSED test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_replicatedmergetree[test_single_shard_multiple_replicas-custom_key_range-key] PASSED test_polymorphic_parts/test.py::test_different_part_types_on_replicas[polymorphic_table_compact-Compact] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-None-True-True] PASSED test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_replicatedmergetree[test_single_shard_multiple_replicas-custom_key_range-sipHash64(key)] PASSED test_polymorphic_parts/test.py::test_different_part_types_on_replicas[polymorphic_table_wide-Wide] PASSED test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_replicatedmergetree[test_single_shard_multiple_replicas-custom_key_sampling-key] PASSED test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-101-100000-SELECT sum(key) FROM {table_name}] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-depends_on1-False-False] PASSED test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_replicatedmergetree[test_single_shard_multiple_replicas-custom_key_sampling-sipHash64(key)] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-depends_on1-False-True] PASSED test_profile_events_s3/test.py::test_profile_events PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-depends_on1-True-False] PASSED test_keeper_restore_from_snapshot/test_disk_s3.py::test_recover_from_snapshot_with_disk_s3 PASSED test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-11-1000-SELECT sum(key) FROM {table_name}] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-depends_on1-True-True] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-None-False-False] PASSED test_polymorphic_parts/test.py::test_polymorphic_parts_basics[first_node0-second_node0] PASSED test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-11-10000-SELECT sum(key) FROM {table_name}] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-None-False-True] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-None-True-False] PASSED test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-11-100000-SELECT sum(key) FROM {table_name}] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-None-True-True] PASSED test_keeper_snapshot_small_distance/test.py::test_snapshot_and_load PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-depends_on1-False-False] PASSED test_parallel_replicas_increase_error_count/test.py::test_increase_error_count PASSED test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-101-1000-SELECT sum(key) FROM {table_name}] PASSED test_match_process_uid_against_data_owner/test.py::test_different_user PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-depends_on1-False-True] PASSED test_polymorphic_parts/test.py::test_polymorphic_parts_basics[first_node1-second_node1] PASSED test_polymorphic_parts/test.py::test_polymorphic_parts_index PASSED test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-101-10000-SELECT sum(key) FROM {table_name}] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-depends_on1-True-False] PASSED test_polymorphic_parts/test.py::test_polymorphic_parts_non_adaptive PASSED test_log_levels_update/test.py::test_log_levels_update PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-depends_on1-True-True] PASSED test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-101-100000-SELECT sum(key) FROM {table_name}] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-None-False-False] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-None-False-True] PASSED test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-11-1000-SELECT sum(key) FROM {table_name}] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-None-True-False] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-None-True-True] PASSED test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-11-10000-SELECT sum(key) FROM {table_name}] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-depends_on1-False-False] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-depends_on1-False-True] PASSED test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-11-100000-SELECT sum(key) FROM {table_name}] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-depends_on1-True-False] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-depends_on1-True-True] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-None-False-False] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-None-False-True] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-None-True-False] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-None-True-True] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-depends_on1-False-False] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-depends_on1-False-True] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-depends_on1-True-False] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-depends_on1-True-True] PASSED test_log_query_probability/test.py::test_log_quries_probability_two PASSED test_grant_and_revoke/test_without_table_engine_grant.py::test_table_engine_and_source_grant SKIPPED [1] test_merge_tree_check_part_with_cache/test.py:28: Skip with debug build and sanitizers. This test manually corrupts cache which triggers LOGICAL_ERROR and leads to crash with those builds ============= 1 failed, 98 passed, 1 skipped in 525.81s (0:08:45) ============== Traceback (most recent call last): File "/home/ubuntu/_work/ClickHouse/ClickHouse/tests/integration/./runner", line 492, in subprocess.check_call(cmd, shell=True, bufsize=0) File "/usr/lib/python3.10/subprocess.py", line 369, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command 'docker run --rm --name clickhouse_integration_tests_94hpeb --privileged --dns-search='.' --memory=30709030912 --security-opt seccomp=unconfined --cap-add=SYS_PTRACE --volume=/home/ubuntu/_work/_temp/test/build/clickhouse:/clickhouse --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/programs/server:/clickhouse-config --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/tests/integration:/ClickHouse/tests/integration --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/utils/backupview:/ClickHouse/utils/backupview --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/utils/grpc-client/pb2:/ClickHouse/utils/grpc-client/pb2 --volume=/run:/run/host:ro --volume=clickhouse_integration_tests_volume:/var/lib/docker -e DOCKER_DOTNET_CLIENT_TAG=11de0b29a15d -e DOCKER_HELPER_TAG=5dc43a6382f0 -e DOCKER_BASE_TAG=5ccda723c1fc -e DOCKER_KERBEROS_KDC_TAG=9391ecdee8d7 -e DOCKER_MYSQL_GOLANG_CLIENT_TAG=9bec2a638e6e -e DOCKER_MYSQL_JAVA_CLIENT_TAG=766bff31cfe4 -e DOCKER_MYSQL_JS_CLIENT_TAG=41ba7c2ec2a1 -e DOCKER_MYSQL_PHP_CLIENT_TAG=88be89c1e3b6 -e DOCKER_NGINX_DAV_TAG=b55ac9cd7519 -e DOCKER_POSTGRESQL_JAVA_CLIENT_TAG=a4eff5c7f4d6 -e DOCKER_PYTHON_BOTTLE_TAG=d862517635bf -e DOCKER_CLIENT_TIMEOUT=300 -e COMPOSE_HTTP_TIMEOUT=600 -e PYTHONUNBUFFERED=1 -e PYTEST_ADDOPTS="--dist=loadfile -n 10 -rfEps --run-id=0 --color=no --durations=0 --report-log=parallel1_0.jsonl --report-log-exclude-logs-on-passed-tests test_executable_dictionary/test.py::test_executable_input_bash test_executable_dictionary/test.py::test_executable_input_python test_executable_dictionary/test.py::test_executable_input_send_chunk_header_python test_executable_dictionary/test.py::test_executable_input_signalled_python test_executable_dictionary/test.py::test_executable_input_slow_python test_executable_dictionary/test.py::test_executable_input_sum_python test_executable_dictionary/test.py::test_executable_non_direct_input_bash test_executable_dictionary/test.py::test_executable_source_argument_python test_executable_dictionary/test.py::test_executable_source_exit_code_check test_executable_dictionary/test.py::test_executable_source_python test_executable_dictionary/test.py::test_executable_source_updated_python test_format_avro_confluent/test.py::test_select test_format_avro_confluent/test.py::test_select_auth test_format_avro_confluent/test.py::test_select_auth_encoded test_format_avro_confluent/test.py::test_select_auth_encoded_complex test_grant_and_revoke/test_without_table_engine_grant.py::test_table_engine_and_source_grant test_groupBitmapAnd_on_distributed/test_groupBitmapAndState_on_distributed_table.py::test_groupBitmapAndState_on_distributed_table test_keeper_profiler/test.py::test_profiler test_keeper_reconfig_remove_many/test.py::test_reconfig_remove_2_and_leader test_keeper_restore_from_snapshot/test_disk_s3.py::test_recover_from_snapshot_with_disk_s3 test_keeper_snapshot_small_distance/test.py::test_snapshot_and_load test_kerberos_auth/test.py::test_bad_path_to_keytab test_kerberos_auth/test.py::test_kerberos_auth_with_keytab test_kerberos_auth/test.py::test_kerberos_auth_without_keytab test_log_levels_update/test.py::test_log_levels_update test_log_query_probability/test.py::test_log_quries_probability_one test_log_query_probability/test.py::test_log_quries_probability_two test_match_process_uid_against_data_owner/test.py::test_different_user test_max_authentication_methods_per_user/test.py::test_alter test_max_authentication_methods_per_user/test.py::test_alter_default_setting test_max_authentication_methods_per_user/test.py::test_create test_max_authentication_methods_per_user/test.py::test_create_default_setting test_merge_tree_check_part_with_cache/test.py::test_check_part_with_cache 'test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_distributed[test_multiple_shards_multiple_replicas-custom_key_sampling-key]' 'test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_distributed[test_multiple_shards_multiple_replicas-custom_key_sampling-sipHash64(key)]' 'test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_distributed[test_single_shard_multiple_replicas-custom_key_sampling-key]' 'test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_distributed[test_single_shard_multiple_replicas-custom_key_sampling-sipHash64(key)]' 'test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_mergetree[test_single_shard_multiple_replicas-custom_key_range-key]' 'test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_mergetree[test_single_shard_multiple_replicas-custom_key_range-sipHash64(key)]' 'test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_mergetree[test_single_shard_multiple_replicas-custom_key_sampling-key]' 'test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_mergetree[test_single_shard_multiple_replicas-custom_key_sampling-sipHash64(key)]' 'test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_replicatedmergetree[test_single_shard_multiple_replicas-custom_key_range-key]' 'test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_replicatedmergetree[test_single_shard_multiple_replicas-custom_key_range-sipHash64(key)]' 'test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_replicatedmergetree[test_single_shard_multiple_replicas-custom_key_sampling-key]' 'test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_replicatedmergetree[test_single_shard_multiple_replicas-custom_key_sampling-sipHash64(key)]' test_parallel_replicas_increase_error_count/test.py::test_increase_error_count 'test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-101-1000-SELECT sum(key) FROM {table_name}]' 'test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-101-10000-SELECT sum(key) FROM {table_name}]' 'test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-101-100000-SELECT sum(key) FROM {table_name}]' 'test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-11-1000-SELECT sum(key) FROM {table_name}]' 'test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-11-10000-SELECT sum(key) FROM {table_name}]' 'test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-11-100000-SELECT sum(key) FROM {table_name}]' 'test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-101-1000-SELECT sum(key) FROM {table_name}]' 'test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-101-10000-SELECT sum(key) FROM {table_name}]' 'test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-101-100000-SELECT sum(key) FROM {table_name}]' 'test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-11-1000-SELECT sum(key) FROM {table_name}]' 'test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-11-10000-SELECT sum(key) FROM {table_name}]' 'test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-11-100000-SELECT sum(key) FROM {table_name}]' test_password_constraints/test.py::test_complexity_rules test_password_constraints/test.py::test_default_password_type test_polymorphic_parts/test.py::test_compact_parts_only 'test_polymorphic_parts/test.py::test_different_part_types_on_replicas[polymorphic_table_compact-Compact]' 'test_polymorphic_parts/test.py::test_different_part_types_on_replicas[polymorphic_table_wide-Wide]' 'test_polymorphic_parts/test.py::test_polymorphic_parts_basics[first_node0-second_node0]' 'test_polymorphic_parts/test.py::test_polymorphic_parts_basics[first_node1-second_node1]' test_polymorphic_parts/test.py::test_polymorphic_parts_index test_polymorphic_parts/test.py::test_polymorphic_parts_non_adaptive test_profile_events_s3/test.py::test_profile_events 'test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-None-False-False]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-None-False-True]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-None-True-False]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-None-True-True]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-depends_on1-False-False]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-depends_on1-False-True]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-depends_on1-True-False]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-depends_on1-True-True]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-None-False-False]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-None-False-True]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-None-True-False]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-None-True-True]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-depends_on1-False-False]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-depends_on1-False-True]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-depends_on1-True-False]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-depends_on1-True-True]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-None-False-False]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-None-False-True]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-None-True-False]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-None-True-True]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-depends_on1-False-False]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-depends_on1-False-True]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-depends_on1-True-False]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-depends_on1-True-True]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-None-False-False]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-None-False-True]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-None-True-False]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-None-True-True]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-depends_on1-False-False]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-depends_on1-False-True]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-depends_on1-True-False]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-depends_on1-True-True]' -vvv " altinityinfra/integration-tests-runner:226bfaf75ac1 ' returned non-zero exit status 1.